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Abstract 

This is a the first in a series of two articles devoted to the question of local 
solvability of doubly characteristic differential operators L, defined, say, in an open 
set tt C M n . 

Suppose the principal symbol pk of L vanishes to second order at (xo>£o) e 
T*Q \ 0, and denote by Qt-c the Hessian form associated to pf. on T^ XQ ^T*Q. As 
the main result of this paper, we show (under some rank conditions and some mild 
additional conditions) that a necessary condition for local solvability of L at xq is 
the existence of some 9 € M such that Re (e t0 Qn) > 0. 

We apply this result in particular to operators of the form 
m 

(0.1) L = OLjk(x)XjXk + lower order terms , 

j,k=l 

where the Xj are smooth real vector fields and the ctjf. are smooth complex coeffi- 
cients forming a symmetric matrix A(x) := {&jk(x)}j,k- We say that L is essentially 
dissipative at xq, if there is some 9 £ M. such that e l8 L is dissipative at xq, in the 
sense that Re (e t6 A(xo)) > 0. For a large class of doubly characteristic operators L 
of this form, our main result implies that a necessary condition for local solvability 
at xo is essential dissipativity of L at xq. 

By means of Hormander's classical necessary condition for local solvability, the 
proof of the main result can be reduced to the following question: 

Suppose that Qa and Qb are two real quadratic forms on a finite dimensional 
symplectic vector space, and let Qc ■= {Qa,Qb} be given by the Poisson bracket 
of Qa and Qb- Then Qc is again a quadratic form, and we may ask: When can we 
find a common zero of Qa and Qb at which Qc does not vanish? 

The study of this question occupies most of the paper, and the answers may be 
of independent interest. 

In the second paper of this series, building on joint work with F. Ricci, M. Peloso 
and others, we shall study local solvability of essential dissipative left-invariant oper- 
ators of the form (jO.lj) on Heisenberg groups in a fairly comprehensive way. Various 
examples exhibiting a kind of exceptional behaviour from previous joint works, e.g., 
with G. Karadzhov, have shown that there is little hope for a complete character- 
ization of locally solvable operators on Heisenberg groups. However, the "generic" 
scheme of what rules local solvability of second order operators on Heisenberg groups 
becomes evident from our work. 1 
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1 Introduction 

Consider a linear differential operator of order k with smooth coefficients 

\a\<k 

d Y' f <) 

on an open subset f2 of R n , where D 



2ni dx\) \ 2ni dx n 
L is said to be locally solvable at xq G fl, if there exists an open neighborhood U of xq such 
that the equation Lu = f admits a distributional solution u G V'{U) for every / G C^°(U) 
(for a slightly more general definition, see 0j). 

Around 1956, Malgrange and Ehrenpreis proved that every constant coefficient oper- 
ator is locally solvable, and shortly later H. Lewy produced the following example of a 
nowhere solvable operator on IR 3 : 

Z = X-iY, where X := #- - * * Y := #- + *± . 

ox 2ou ay 2 ou 

Not quite incidentally, Z is a left-invariant operator on a 2-step nilpotent Lie group, the 
Heisenberg group Mj. 

This example gave rise to an intensive study of so-called principal type operators, 
which eventually led, most notably through the work of Hormander, Maslov, Egorov, 
Nirenberg-Treves and Beals-Fefferman, to a complete solution of the problem of local 
solvability of such operators (see |3|). 

Let us recall some notation. Denote by Pk(x,£) '■= J2\ a \=k c »( x )£ a the principal symbol 
of L. We shall consider pk as an invariantly defined function on the reduced cotangent 
bundle C := T*tt \ = x (R n \ {0}) of Q. 

Let us denotes by 7i"i the base projection 7Ti : T*Q — > Q, (x,£) i— > x. T*Q carries a 
canonical 1-form, which, in the usual coordinates, is given by a = YTj=i^j^ x ji so that 
T*Q has a canonical symplectic structure, given by the 2-form a :— da — J^™ =1 d£j A dxj. 

keywords: linear partial differential operator, local solvability, doubly characteristic, real quadric, 
Poisson bracket 
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In particular, for any smooth real function a on f2, its corresponding Hamiltonian vector 
field H a is well-defined, and explicitly given by 

H f da d da d 

If 7 is an integral curve of H a , i.e., if 4f'j(t) = H a (y(t)), then a is constant along 7, and 7 
is called a null bicharacteristic of a, if a vanishes along 7. Finally, the (canonical) Poisson 
bracket of two smooth functions a and b on T* (O) is given by 



{a, 6} :=a(H a7 H b ) = H a b = Y, 

3=1 



da db da db 



d£j dxj dxj i)£ 



Let 

s = h = o}cc 

denote the characteristic variety of L. L is said to be of principal type, if -0^2 does not 
vanish on £ (or, more generally, if for every ( G £ there is a real number # such that 
d(Re (e ld pk))(C) an d «(C) are non-proportional). 

In 1960, Hormander proved the following fundamental result on non-existence of so- 
lutions (see [3]): 

Theorem 1.1 (Hormander) Suppose there is some £0 £ K n \ {0} such that 

a(x , f ) = K^o, Co) = and {a, 6}(x , Co) 7^ 0, 
where a := Rep/j and b := Impk- Then L is not locally solvable at xq. 

A complete answer to the question of local solvability of principal type operators L 
was eventually given in terms of the following condition (V) of Nirenberg and Treves: 

(V). The function Im (e ie pk) does not take both positive and negative values along a 
null-bicharacteristic 70 (t) of Re {e l8 pk), for any 9 e R. 

In fact, L of principal type is locally solvable at Xq if and only if (V) holds over some 
neighborhood of Xq. Notice that this is a condition solely on the principal symbol of L. 

In this article, we shall consider differential operators L with double characteristics. 

Let 

S 2 :={(x,0GC: dp k (x,Z) = 0}. 

denote the set of double characteristics of L. By Euler's identity, £2 is contained in the 
characteristic variety E. 

In order to formulate our main theorem, we need to introduce some further notation 
concerning quadratic forms. 

If A e Sym (n, K), we shall denote by Qa the associated quadratic form 

Q A (z) := *zAz, z G I", 

on K n . For any non-empty subset M of a IK- vector space V, spanuM will denote its linear 
span over IK in V. 
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Assume for a moment that V is a finite dimensional real vector space, endowed with 
a symplectic form uj. If Q is a complex-valued quadratic form on V, we shall often view it 
as a symmetric bilinear form on the complexification V c of V, and shall denote by Q(v) 
the quadratic form Q(v,v). Q and u then determine a linear endomorphism S of V c by 
imposing that 

u(u, Sv) = Q(u, v). 

Then, S G sp(V c ,u), i.e., 

u(Sv, w) + u>(v, Sw) = 0. 

5 is called the Hamilton map of Q. We shall then also write Q = Q s . Clearly, S is real, 
i.e., S G sp(V,u), if Q is real. 

Recall also that we can associate to any smooth function a on V the Hamiltonian 
vector field H% such that uj(H^, Y) = da{Y) for all vector fields Y on V, and define the 
associated Poisson bracket accordingly by 

{a,b} u :=u>(HZ,H?). 

Let us endow V with the Poisson bracket associated to u, and denote by Q(V) the 
space of all complex symmetric quadratic forms on V. One easily computes that 

(1.2) {Q Sl ,Q S2 h = Q- 2[Sl ' S2] , S U S 2 G sp(V c ,tu), 

which proves the well-known fact that ^Q(V), {•, is a Lie algebra, isomorphic to 
sp(V c ,u) under the isomorphism Q s t— > —2S. 

Consider now again our differential operator L. If (x,£) G S 2 , then we denote by 
Q — Q(x,£) '■= QD 2 Pk (x£) the associated Hessian form on T^ X ^C — M 2 ™, and by S = S^ x ^ 
the corresponding Hamilton map, given by 

a(u, Sv) = Q(u, v). 

Let A, B G Sym(m, R). We say that A, B form a non-dissipative pair, if is the 
only positive-semidefinite element in span R {A, B}. Notice that this is equivalent to the 
following statement: 

There is no 6 G R such that Re (e ie {A + iB^j > 0. 

Moreover, we put 

maxrank {A, B} := maxjrankF : F G spanK{A, B}} 
minrank{A,5} := minjrankF : F G span R {A, B} ,F ^ 0}. 

Notice that minrank {A, B} > 2 for a non-dissipative pair A, B. 
We can now state our main result. 

Theorem 1.2 Let (x ,£ ) G S 2 , and put H := D 2 p k (x ,£ ) = A + iB, with A, B G 
Sym (2n, R). Define C G Sym (2n, R) by 

Qc '■= {Qa, Qb}, 

and denote by Q H the Hessian form of L at (xq,^ ). Assume that 



4 



(a) There is no 9 el such that Re(e Qh) > 0, i.e., A,B form a non-dissipative pair. 



(b) The matrices A, B and C are linearly independent over R. 

(c) Either 

(i) minrank {A, B} > 3 and maxrank {A, B} > 17, or 

(ii) minrank {A, B} = 2, maxrank {A, B} > 9, and the joint kernel ker Anker B of 
A and B is either trivial, or a symplectic subspace with respect to the canonical 
symplectic form a. 

Then L is not locally solvable at Xq. 

Remarks 1.3 ( i) Related results for a rather particular class of operators on Heisenberg 
groups have been given in [Tj. The proof in that article was specific to the class under 
consideration and could not be extended, so that the proof of Theorem 11.21 is completely 
different. 

(ii) If S = S\ + 2S2 denotes the Hamilton map associated to L at (xo, £0), then, in view 
of (JH2J), condition (b) is equivalent to requiring that Si, S2 and the commutator [Si, S2] 
are linearly independent, a mild condition which is satisfied "generically". 

(iii) We do not know if the conditions on maxrank {A, B} in (c) are optimal, but 
various examples of left-invariant differential operators on Heisenberg groups (see, e.g., pj, 
[7j) show that the statement of the theorem is definitely wrong, if maxrank {A, B} < 6. 
Compare also the counter-examples to Theorem 11.51 on which the proof is based, in 
Remarks 12.71 13.61 which also indicate that the condition on the joint kernel of A and B is 
indispensible. There are surely obstructions of topological respectively geometric nature 
if the ranks are too small, and the counter-examples that we know so far indicate that a 
comprehensive answer to the question when the conclusion of the theorem will hold would 
require a rather tedious case to case study of lower rank situations. 

(iv) The main condition in the theorem is condition (a), which, like condition (V), is 
again a sign condition on the principal symbol of L. 

Let us illustrate the theorem for second order operators of the form 



j,k=i 

where Xi, . . . , X m are smooth real vector fields and where A(x) := {®jk(x)}j,k €= Sym (m, C) 
is a complex matrix varying smoothly in x. We then write 



m 



(1.3) 




lower order terms , 



A(x) = A(x) +iB(x), xeQ, 



with A(x), B(x) G Sym (m, R). Denote by 2iriqj the symbol of Xj, put q := \qi, . . . , q m ) 
and define the skew-symmetric m x m- matrix 



J {*,o ■= {{<n>qk}{x,Z)) jtk= 



Since 



5 



we have 



{(z,OeC:g(z,0 = 0}cE 2 , 

and equality holds here, if A(x) is non-degenerate. Notice also that if A(x ) is non- 
degenerate and q(xo, £o) = 0, then J( Xo ^ ) * s non-degenerate if and only if E 2 is symplectic 
in a neighborhood of (xoj£o) ( see ; e -g- [I2J, Proposition 3.1, Ch. VII). 

Let us assume that Jf x ,g) is non-degenerate. Then we can associate to J(x,g\ the skew 
form 

W(xA)(v, w) := % XJ^))' 1 w, v,we R m , 

which defines a symplectic structure on IR m , with associated Poisson structure {•, -}( x ,o- 
In particular, m = 2d is even. 

Corollary 1.4 Let L be given by (II. 3j) . and let x G f2. Assume that 

(a) A := A(xo),-B := -B(xo) /orm a non-dissipative pair. 

(b) There exists some £o £ ^ n \ {0} such that q(xQ,£o) = 0, J(x ,€o) ^ s non-degenerate, 
and the matrices A,B and C := C(x ,£, ) are linearly independent overM., where 
C{x ,^ ) G Sym(m,M) is defined by 

Qc(x Q £ ) ■= {Qa,Qb}(x ,£ ) ■ 

(c) A and B satisfy the conditions (c) in Theorem \l. 6 A only with the canonical symplectic 
structure a on M. 2n replaced by the symplectic structure ur xo £ \ on R m . 

Then L is not locally solvable at x . 

Corollarv ll .41 shows that a "generic" operator L of the form (jl. 3)1 can be locally solvable 
at xq only if there is some ffel such that Re (e l6 A(xo)) > 0, which means that e L is 
dissipative "at" xq. A major task which remains is thus to study local solvability of L 
under the assumption that A(x) = Re^4(s) > for every x G f2. A stronger condition is 
the condition 

(1.4) \B(x)\<A(x), xeQ. 

This condition is equivalent to Sjostrand's cone condition ^T]. It implies hypoellipticity 
with loss of one derivative of the transposed operator *L, for "generic" first order terms 
in (II .3j) . and thus local solvability of L at Xq (see [1], Ch. 22.4, for details and further 
references) . 

Since, however, local solvability of L is in general a much weaker condition than 
hypoellipticity of *L, we are still rather far from understanding what rules local solvability 
in general, even when the cone-condition is satisfied. 

Nevertheless, for the case of homogeneous, left-invariant second order differential op- 
erators on the Heisenberg group H n , a rather complete answer had been given in jU], 
and in the sequel to the present article, we shall extend these results by dropping the 
cone condition, thus giving a fairly comprehensive answer for left-invariant operators on 
Heisenberg groups. 
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We should like to mention that, even if the cone- condition is satisfied, for instance 
small perturbations of the coefficients of the first order terms preserving the values at 
Xo, may influence local solvability and lead to local solvability in situations where the 
unperturbed operator is not locally solvable at x (see, e.g., 0). Moreover, if, e.g., 
maxrank {A(xq), B(xq)} = 4 or 6, then the conclusion in Theorem 11.21 mav not be true 
(see 0,0). 

All these results indicate that there is rather little hope for a complete characterization 
of local solvability for doubly characteristic operators in general, but that Theorem 11.21 
in combination with the above mentioned results on hypoellipticity give at least rather 
satisfactory answers in the "generic" case. 

Theorem 11.21 can be reduced by means of Hormander's Theorem II. II to the following 
result concerning real quadrics, which may also be of independent interest and which 
represents the core of this work. 

Theorem 1.5 Assume that M. n = M 2d is endowed with the canonical symplectic form, 
and let A,B e Sym (n, M), forming a non-dissipative pair. Define Qc '■= {Qa, Qb} as the 
Poisson bracket of Qa and Qb, and assume that A,B and C are linearly independent. 
Then there exists a point x G lR n such that 

Qa(x) = Qb(x) = and Q c (x) ^ 0, 

provided one of the following conditions are satisfied: 

(i) minrank {A, B} > 3 and maxrank {A, B} > 17; 

(ii) minrank {A, B} = 2, maxrank {A, B} > 9, and the joint radical TZa,b '■= ker A fl 
ker B of Qa and Qb is either trivial, i.e., kei A fl keri? = {0}, or a symplectic 
subspace ofM. n . 

The article is organized as follows. Sections 2 and 3 are devoted to the proof of 
Theorem 11.51 Notice that this theorem essentially states that the quadratic form Qc '■ = 
{Qa,Qb} can only vanish on the joint zero set {Qa = 0} fl {Qb = 0} of two linearly 
independent quadratic forms Qa and Qb forming a non-dissipative pair, if C is a linear 
combination of A and B. 

Of course, this can only be true if {Qa = 0}H{Qb = 0} is sufficiently big, and we shall 
show in Section 2 that (under these assumptions ) the quadrics {Qa = 0} and {Qb = 0} 
do in fact intersect tranversally in a variety M of dimension n — 2. 

In Section 3.1, we prove some auxiliary results and recall some basic notions and facts 
on semi-algebraic sets. 

The proof of Theorem ll.5l is then given in Sections 3.1 and 3.2. We distinguish between 
the situation where no stratum of M spans lR ra (Section 3.1) and the case where at least 
one stratum spans (Section 3.2). It is interesting to notice that the condition that Qc be 
the Poisson bracket of Qa and Qb is only needed in the first case (see Theorems 13.51 and 
I3.12|) . We also present a number of examples in order to demonstrate that the conditions 
in the main Theorem 13.51 of Section 3.1 are essentially necessary. 

Section 4 finally contains the proof of Theorem 1 1 . 21 and Corollarv ll.4l The main idea is 
to find a simply characteristic point in the vicinity of a given doubly-characteristic point 
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at which Hormander's condition is satisfied. Moreover, we give various applications of this 
theorem to left-invariant differential operators on 2-step nilpotent Lie groups (compare 
Corollary 14. II for general 2-step nilpotent Lie groups, and Corollary 14.21 for the particular 
case of the Heisenberg group). We also indicate that Corollary 11.41 has applications to 
higher step situations too, for instance on r-step nilpotent Lie groups. 

2 On the intersection of two real quadrics 

If V is a K- vector space, and if v\, . . . , V}~ are vectors in V, then v\ A • • • A will denote 
their exterior product in A k (V). In particular, v\,...,Vk are linearly dependent if and 
only if v\ A • • • A Vk — 0. The open interior of a subset S of some topological space will be 
denoted by S°. 

If M is a non-empty subset of Sym (n, R), then we say that M is non-dissipative, if 
is the only positive-semidefinite element in spangM. 

Lemma 2.1 Let M be a non-empty subset o/Sym(n, R). Then the following are equiv- 
alent: 

(i) M is non-dissipative. 

(ii) There is some positive definite matrix Q > such that 

(2.1) tr ( l QFQ) = for every F G M. 

(Hi) There is a matrix T G GL (n, R) such that 

(2.2) tr ( *TFT) = for every F G M. 

Proof, (i) =>- (ii). Let V = Sym (n, R), and let V C V denote the closed cone of positive 
semidefinite matrices in V. Put K := {E 6 V : tr E = 1}. Notice that E G V has 
vanishing trace if and only if E = 0. Therefore, V \ {0} = Ut>o 

If W := span^M, then K and W are convex subsets of V, which are disjoint, by (i). 
Moreover, K is compact and W is closed. By Hahn-Banach's theorem (see, e.g. [TO], 
Theorem 3.4 (b)), there exists a linear functional \l G V* and 7 G R, such that 

12(E) > 7 > fi(F) for all EeK,F eW. 

Since W is a linear space, this implies n\w = 0, hence /i(-E') > for all E e K. 
Choose ? 6 F such that /i(M) = tr (FM) for all M e V. Then 

tr (PE) > for all E eV. 

Rotating coordinates, if necessary, we may assume that P is diagonal, say P = diag(Aj). 
But then clearly Aj > 0, j = 1, . . . ,n, hence P > 0. Choose Q > such that P = Q 2 . 
Then 

= tr (Q 2 F) = tr (*QFQ) for all F eW, 

so that (j2UJ holds, 
(ii) =>■ (iii) is trivial. 
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(iii) =>- (i). Assume that F G W and F > 0. Then (iii) implies that tr (tTFT) = 0, where 
*TFT > 0. This implies *TFT = 0, hence F = 0. 

Q.E.D. 

If A G Sym (n, R), then we put 

T A := {z G R n : Q A (z) < 0} . 

Recall that a pair A,B G Sym (n, R) such that {A, 5} is non-dissipative is called a 
non-dissipative pair. Notice that this property depends only on the linear span of A and 
B. In view of Lemma \2. 11 it will sometimes be convenient to assume that a linear change 
of coordinates has been performed so that A, B have vanishing trace. 

By S 11 ^ 1 we shall denote the Euclidean unit sphere in IR n , and by B r (x) the open 
Euclidean ball of radius r centered at x G W 1 . 

Theorem 2.2 Let A,Be Sym(n, R), and assume that 

(2.3) tiA = trB = 
and 

(2.4) r A c r B . 

Then there is some eel such that 

(2.5) B = cA. 

Proof. After a rotation of coordinates, we may assume that 

{ -A 2 

w.r. to the decomposition IR ra = M. k x WL e , with (diagonal) matrices A\ > 0, A 2 > 0. Write 
correspondingly 

B\ B 3 



B 



t B% —Bo 



We decompose z G lR n as z = (x, y) G M. h x 

The case B = is trivial, so let us assume that B ^ 0. Observe first that 

5 2 > 0, 

for, if y G then (0, y) G T A , hence - fy-B 2 y < 0, by ■ 
We claim that, for every y G R , 

(2.6) a(j/) := W > =S- f3{y) := t yB 2 y > 0. 

Indeed, if Q := {y G R : a(y) > 0}, and if y G fi, then, given x G there is some 
r x > such that (x, ty) G r A whenever t G 1, \t\ > r x . We thus find that 

l xB x x + 2t(B 3 y) ■ x - t 2 f3(y) < provided \t\ > r x . 
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If f3{y) = 0, choosing both signs of t, we see that this implies 

l xB x x < and (B 3 y) ■ x = for all x G R k , 

hence —B\ > and B 3 y = 0. Therefore, the matrix 

B x 
-B 2 

is postive semi-definite. Since it has vanishing trace, it must vanish, so that B\ = 0, B 2 = 
0, and B 3 y = 0. 

Thus, if P(yo) = for some yo G Q, then (3(y) = and consequently £>3?/ = 0, for 
every y € Q. Since f2 is dense in R , we obtain .83 = 0, hence 5 = 0, contradicting our 
assumption on B. This proves ([2.6)1 . 
For eeS n := S^ 1 HflcK', put 

Q e A {x) := Qa(x, e) = l xA\x — a(e) , 

where a(e) = fe^e > . Similarly, let 

Q%{x) := Q B {x, e) = l xB x x + 2{B 3 e) ■ x - /3(e) , 

where (3(e) = *e£>2e > 0, because of ([2.6)1 . 
Finally, for e G 5*0 fixed, let 

/(x) := <5a( 2; ) — *x— r^-x — 1 , 

a(e) a(e) 

1 B 

/3(e) /3(e) 

where £ := 2j^. From ()2.4jl . we know that 

(2.7) /(x) < =^ #(x) < for all x G 

Lemma 2.3 Lei A,B& Sym (fc, R), and assume that A > 0. Moreover, let £ G M fe , and 

/(x) := *xAr - 1, 

g(x) := *x£>x + £ • x — 1 . 

T/ien (|2.7|) implies 

(2.8) tr.A>trB. 
Moreover, either f = g, or 

(2.9) tr.A>trB. 
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Proof. Observe that 

(2.10) Af = 2trA, Ag = 2tiB. 

Assume now that tr A < tr B. 
Then, by (|2.1()|) . A(g — f) > 0, so that g — f is subharmonic. Moreover, g(x) — f(x) < 
for f(x) = 0. By the maximum principle, we thus conclude that 

g(x) — f(x) < in the ellipsoid {/ < 0} . 

Thus, there is some e > 0, so that 

l x[B - A)x + £ ■ x < for all x G B e {0) . 

Then also l x{B — A)x — £ ■ x < 0, hence 

l x{B - A)x < for all x £ B e (0) . 

By homogeneity, this implies B — A < 0, hence 

(2.11) B<A. 

In particular, tr B < trA, hence tr^4 = ti B. But then A — B > 0, tr (A — B) =0, hence 
A - B = 0, so that A = B. 

So, either ti A > ti B, or A = B. This proves ()2.8|) . 
Moreover if A = B, then g(x) = f(x) + and, after a linear change of coordinates, we 
may assume that A = I, i.e., 

/(x) = |x| 2 -l. 

Thus, \x\ < 1 implies \x\ 2 + £ ■ x < 1. If £ 7^ 0, choosing a; = 4|, we obtain 1 + |£| < 1, 
hence £ = 0, a contradiction. Therefore, £ = 0, hence / = g. 



Q.E.D. 



Going back to the proof of Theorem 12.21 we can now conclude that 



a(e) (3(e) 



i.e., 



(2.12) P(e)tiA 1 > a(e)trSi for all e E Sq , 
and this inequality is strict, unless f = g, i.e., 

(2.13) — = 77TT and C = (2//3(e))B s e = . 

Notice that, by continuity and since Sq is dense in S^ -1 , (|2.12j) holds indeed for all 
e G S e -\ 

We distinguish therefore two cases. 
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(a) If there exists some e G Sq such that (|2.12|) holds strictly, we choose an orthonormal 
basis e% = e, e 2 , . . . , &i of 1R £ . Then 

/3(e j )trA 1 > a(e j )trB 1 , j = !,...,£, 

and the inequality is strict for j = 1. Summing in j, we thus obtain 

tr B 2 tr Ai > tr A 2 tr £?i . 
But, since = tr A = trB, we have tr A 2 = tr Ai, tr B 2 = tr B\, hence 

trSitrAi > trA 1 trB 1 , 

a contradiction. 

(b) There remains the case where 

/3(e) Ai = a(e)-Bi and B 3 e = for all e E Sq. 
Again, by continuity, this then holds for all e G £? . But then £? 3 = 0, hence 



B x 
-£? 2 



Moreover, B\ = cA\ for some c > 0. Then we see that 

/3(e) = ca(e) , for all e G S"^ 1 , 
which implies, by homogeneity, 

\}B 2 y = fy(cA 2 )y for all y G R e , 

i.e., £? 2 = cA 2 . We thus get 

B = cA. 

Q.E.D. 

Proposition 2.4 Lei i? and -D be open ellipsoids in IR n ,n > 2, whose boundaries don't 
intersect transversally anywhere. Then, if E R D ^ 0, either E d D or D <Z E . 

Proof. J. Case n = 2. 

We may then assume that the boundary 3D of D is a circle 

0£> = {(x, y)ER 2 :(x- 2 + (y - V) 2 = r 2 } , 

and that 



dE = {(x,y)eR 2 :^ + ^ = l 



where a > b > 0. 

Assume also that neither E C D nor D C E. Then, by convexity, dE can neither be 
contained in D, nor in (IR 2 \ D), i.e., there is a point in dE lying in IR 2 \ D, and another 
one in D, hence, by continuity of the boundary curve of D, dE fl dD ^ 0. 
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Say that E and D pierce at X G dE fl 3D, if every neighborhood U of X contains a 
point in dE \ D and one in dD \ E. 

If E and D don't pierce at any X G dE fl <9-D, again by continuity of the boundary 
curves of E and D, either dE C D or <9/} C -E, hence E G D or D G E, by convexity. 

Consequently, E and .D pierce at at least one point X G dE fl dD. By symmetry, we 
may assume that X = (x , y ), with x > and y > 0. 

Assume first < x < a, hence < y < b. If D and E pierce at X, then <9.D and 
dE must have the same curvature at X, namely 1/r, as can easily bee seen from the 
local Taylor expansions of the boundaries curves. If a = b, then this implies D = E, 
so assume a > b. Since dD has constant curvature, there are exactly four points on 
dE which are potential piercing points, namely X ,X 1 : = (x , —yo),X 2 :— (—Xo,yo) and 
X 4 := (— x ,— y ). Moreover, by continuity, there must be at least one more piercing 
point, besides X . 

Assume, e.g., that Xi is a second piercing point. Since u := (ff, fl) is normal to 



Moreover, computing the curvature of the ellipse dE at X , and comparing it with 
that of the circle dD, we then find that 



The case where X 3 is a second piercing point can be treated in a similar way. And, 
X 4 cannot be a piercing point, since X 4 = — X , so that the center of dD would have to 
ly on the two parallel lines X + lRz/ and — X Q + R(— v ), which would imply u = sX 
for some s G R. But this is impossible, since a > b. 

The cases where xo = or xo = a are even easier, as well as the case where a = b, and 
are left to the reader. 

2. Case n > 3. 

If neither E G D nor D G E, we may choose points x G E fl D, X\ G E \ D and 
x 2 G D\E spanning a two-dimensional afline plane V. We may then reduce the problem 
to the 2-dimensional case by restricting ourselves to the afline plane V. 

Indeed, if E := {P < 0}, D := {Q < 0}, for suitable elliptic quadratic functions P 
and Q, then let 

p(s,t) := P(x + s(x 1 - x ) + t(x 2 - x )) = P o 7(s,t) , 

q(s,t) := Q(x + s(xi - x ) +t(x 2 - x )) = Q o >y(s,t), (s,t) e M 2 . 

Then p and q are elliptic quadratic functions on M 2 , and 





b/a 2 



1/6 




hence = (1 — ^)(1 — ^). This implies x 



a, in contradiction to our assumptions. 



E (IV = {p < 0} =: E, D n V = {q < 0} =: D , 
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in the coordinates (s, t) for V. 

And, if p(s, t) = = q(s, t), then Vp(s, t) A Vg(s, t) = 0. For, if Vp(s, t) and Vg(s, t) 
were linearly independent, then for X = j(s,t) G dE fl <9P, the vectors 

VP(X)-P 7 (s,t) = Vp(M) 
VQ(X)-P 7 ( S ,t) = Vg(s,t), 

would be linearly independent, hence also VP(X) A VQ(X) 7^ 0. 

Thus, we could apply Case 1 to E and D, and conclude that one is contained in the 
other, say, e.g., E C P. But this would contradict our assumption that X\ G EC\V\DC\V . 

Q.E.D. 

We shall need the following modification of Proposition 12.41 

Proposition 2.5 Let A, B G Sym(n,R), A > 0,P > 0, suc/i i/iai rank A > 2,rankP > 
2, and let 

D := {x G R n : Qa(^) < 1}, E := {x G R n : Q B (x) < 1}. 

// t/ie boundaries of D and E don't intersect transversally anywhere, and if E fl D 7^ ; 
t/ien either E C D or D G E. 

Proof. We can argue similarly as in the proof of the case n > 3 of Proposition 12.41 

Let TZa = ker A and TZb = kerP denote the radical of Qa and Qb- Since rank A > 2, 
we can find vectors £1,^2 £ R n such that £1 A £ 2 7^ and P.^ C £f fl <^~, and similarly 
fi, | 2 e M n such that fi A | 2 ^ and TZ B C £f n 

If neither P C P nor P C P, we may choose points £0 = G P fl P, x\ E E \ D 
and X2 G P \ P spanning a two-dimensional plane V. Choose linearly independent vectors 
771, ... , r/ n _ 2 G lR n such that V = r/ffl- ■ -nr/^_ 2 , and consider u; := £1 A^Ar/x A- ■ ■ Ar/ n _ 2 . If 
u; 7^ 0, then V fl Pa = {0}, so that Qa\v > is a positive definite, i.e., elliptic, quadratic 
form on V. Similarly, if also u := £1 A £ 2 A rji A ■ ■ • A r/ n _ 2 7^ 0, then Qb\v > is elliptic, 
and we can conclude the proof as in Propositior i2.4l 

We shall show that we can slightly vary the points x\ and £2 in order to achieve that 
uj 7^ and Co 0. Indeed, since P \ P 7^ 0, and since the boundary of P is a smooth 
submanifold of codimension 1, P \ P has non-empty interior, and we may assume that 
X\ lies in this interior. Similarly, we may assume that X2 lies in the interior of P \ P. 
Therefore, there is some e > 0, such that X\ + y% G E \ D and X2 + 2/2 G P \ P for all 
2/1)2/2 £ Pe(0). Let us now replace rjj by r/j = rjj + 6j, with 5, G 1R™ sufficiently small, 
in order to achieve that to 7^ 0, to 7^ 0, for the corresponding n-forms co> and u. Then we 

can find y±, y 2 G P e (0) such that fjj(xi + j/i) = = %(x 2 +2/2) for j = 1 , n — 2 (just 

solve the linear equations r)j • y\ = —5j ■ Xi, and rjj ■ y 2 = —5j ■ x 2 , for j — 1, . . . , n — 2.) 
Replacing X\ by x\ + y\ and x 2 by x 2 + y 2 , we can thus assume that to 7^ and u; 7^ 0, 
which completes the proof. 

Q.E.D. 

Let us next extend the previous results to non-semidefinite forms. 
If W is a linear subspace of Sym (n, R) , we say that A £ W has maximal rank in W, 
if rank A = maxrank M. 

For A, B G Sym (n, K) and r > 0, let 

T^:={gA<r}, T B :={Q B <r}. 
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Then in particular = F% F B — F° B . Notice that if z G dT r A , and if Az ^ 0, then dT r A is 
an analytic manifold of codimension 1 near z, and Az is normal to it at z. If such a vector 
z exists, we say that dT r A is a variety of dimension n — 1. If z & dT r A fl dT r B , and if Az 
and Bz are linearly independent, then we say that the boundaries dT r A and dT r B intersect 
transver sally at z. 

Theorem 2.6 Let W C Sym (n, 1R) 6e a non-trivial subspace of Sym (n, M) and r > 0, 
and fix A & W of maximal rank in W, with signature (k,£\). Let also B e W, and assume 
that dY r A and dY r B are varieties of dimension n — 1. 

// t/jere no point in dT r A fl cT^ at which the boundaries dT r A and dT r B intersect 
transver sally, and if || B — A \\ is sufficiently small, then either T r A C or C T^, 
provided k > 2. 

Proof. Let £ := n — k. After rotating the coordinates and scaling in every coordinate, if 
necessary, we may assume without loss of generality that 

with respect to the decomposition M. n = M. k x where, by our assumption, k > 2, and 
where A 2 > has rank £\. 

If i = 0, i.e., if A = /„, and if I? is so close to A that B = B\ > 0, then the statement 
is clear, by Proposition I2.4| if r > 0, and the case r = is trivial. 

So, assume that £ > 1. We also assume that S is so close to A that B\ > 0. 
As in the proof of Theorem 12. 2| we decompose z = (x, y), x G w G E^, and put, 
for y G R e , 

Q y A (x) := Q A (x, y) = \x\ 2 - a{y) , 

Qb(x) ■= Qb(x,v)= t xB 1 x + 2(B 3 y)-x-[3(y), 

where a(y) := \)A 2 y > and f3{y) := S)B 2 y. Notice that we can write 

(2.14) Ql(x) = \x + B A y)B 1 (x + B 4 y) - l yB 2 y, 

with £? 4 := B^ X B 3 and B 2 : = B 2 + l B 3 B^ l B 3 . Observe that rank_B 2 < £i, since A has 
maximal rank in span {A, B}. 

We claim that B 2 > and rank_B 2 = £i, if \\B — A\\ is assumed sufficiently small. 
Indeed, if \\B — A\\ is sufficiently small, then Qb 2 \w > on a space W of dimension 
£i, and thus also Q B2 \w > 0. If we decompose M. e = W © H, then in suitable blocks of 

coordinates (w, h) subordinate to this decomposition, we may write B 2 = 

where D > has rank £\. Then Q B2 (w, h) = \w + Sh)D(w + Sh) — l hFh, for suitable 
matrices S, F, and since rank5 2 < necessarily F = 0. This implies the claim. 

Let Q := R e , if r > 0, and Sl:=R e \ (ker A 2 U ker £ 2 ), if r = 0. Notice that i£ y e Q, 
then r + a(y) > and r + /3(y) := r + > 0, so that the ball 

D y r := {Q\ < r} = {x G R k : |x| 2 < r + a(y)} 



( ^ E ) 
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and the ellipsoid 

El := {Ql < r} 

have non-empty interiors. 

Notice that we can exclude the case where l\ — and r = 0, for then A 2 = = B 2 , 
so that T° A = {0} x R e and T° B = {(x,y) £ R k x R e : x = — B 4 y} have both codimension 
greater or equal to two, so that there is nothing to prove. 

We therefore assume henceforth that t\ > 1, or r > 0. Then fl is dense in IR £ . 

Fix Uq G M. e such that r + a(?/o) > 0. If we choose \\B — A\\ sufficiently small, then we 
may assume that r + t yo(A 2 — ^^B^ 2 B^)y Q > 0. Modifying y slightly, if necessary, we 
may in addition assume that r + i y B 2 yo > 0. But then the point (—-847/0,1/0) lies in the 
open interior of D yo and of E yo . This shows that the set 

Q l :={yeQ: {D^f fl (J5£)° ^ 0} 

is a non-empty, open subset of fl. 
1. Case: t x > 2, or r > 0. 

Then f2 is connected. We prove that then = fi. 

For, otherwise, we find y G fl± and G f2 \ fli. Connect y and y' in by a continuous 
path 7 : [0, 1] — > fl, and put 

A := DJ® , ^ := f^W , t G [0, 1] , 

and 

r := inf{t G [0,1] : 7 (t) G fl \ fl^ . 

Then j(t) G fi \ fii, since f2 \ fli is closed in fl. Now, observe that for y G fii, the ball D r y 
and the ellipsoid don't intersect transversally at common points of their boundaries. 
By Proposition 12.41 since k > 2, we then have 

(2.15) D y r C E», or E v r C D v r , for every y G fli. 
Thus, for some sequence {sj}j in [0, r[ tending to r, we have 

(2.16) D Sj C /•:.,,. ovE Sj GD Sj . 

By continuity, this implies (-D T )° fl (-EV) 7^ 0, a contradiction. 

We have thus shown that the alternative (|2.15|) holds for every y G fl. 
However, if D y C -E^ for every y & fl, then 

QA(x,y)<r QB(x,y)<r for all y G fl . 

By continuity, this remains true also for all y G so that in this case T r A GT r B . Similarly, 
if E~! C D y r for every y E fl, then we find that T r B C P^. 
Assume therefore that there exist y,y' &fl such that 

D» C and £ r y ' C -D^' . 

We claim that we can then find an rj G fl such that 

(2.17) E? = D*. 
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Since fl is connected, we can connect y and y' in by a continuous path 7 : [0, 1] — > Q. 
Put 

A := DJ {t) , E t := E?® , t G [0, 1] . 
Then D C Eb, A C A- Let 

r := inf{t £ [0, 1] : E t C A} . 

We claim that A = A) which proves (|2.17|) . 

Since E t arizes from a fixed centrally symmetric ellipsoid E by scaling with a positive 
factor and translation by some vector, both depending continuously on t (and similarly 
for A)> we clearly have E T C D T . The case r = is then obvious, so assume r > 0. Then 
A C A for s < r, so that A C E T , by continuity. 

With 77 as in (J2.17|) . we have in particular 

(2.18) {Ql = r} = {Ql = r} = {x : \x\ 2 = r + a( V )} . 
This implies 

l xBix + 2(B 3 r}) ■ x — (P(t]) + r) = for every x satisfying \x\ = (r + aij])) 1 ^ 2 . 
Exploiting this for x and —x, we see that 

(2.19) £3*7 = 0, 
and then, by scaling, 

t xBx x = Pyl + r \x\ 2 for all x G R k , 
«(r?) + r 

hence 

(2.20) A = A A 

with /3i := (/?(r/) + r)/{a{rj) + r). 

Let 2; := (x,r)), where x = (r + (^(r?)) 1 / 2 :?/, with x' G S^ -1 . Then 2; G dY r A H <9r^, and 
since is normal to and .Bz normal to cT^ at z, the assumptions in the theorem 
imply that Az A Bz = 0, where, because of ()2.19|) . 

AZ = ( -A 2 rj ) ' 52 = ( 'A* - B 2 r, ) " 
Thus = fiiAz, hence 

(2.21) *B 3 £ - B 2 r] + (3\A 2 r) = whenever \x\ = (r + a(r/)) 1/2 . 

Taking the scalar product with 77, in view of (|2.19j) we get —/3(r]) + Pia(r]) = 0, hence 
r(a(r ] )-P(r l )) = 0. 

If r > 0, this implies a(rj) = /3(r]), hence 
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If r = 0, notice that (3\ is close to 1, in view of (|2.2(J|) and since \\A — B\\ is assumed 
small. Therefore, we may replace B by without loss of generality, so that again we 

may assume that (3\ = 1. 

Moreover, by (|2.21|) . then 

l B 3 x = (B 2 — A 2 )r] whenever |x| = (r + a{j])) 1 ^ 2 . 
Applying this to x and —x, we find that B 3 = and A 2 rj = B 2 rj. We therefore obtain 

where then also B 2 > 0. 
Thus 

Qa(x,v) = \x\ 2 - Qa 2 (v), Qb(x,v) = \x\ 2 - Q B2 (y)- 
Fix x such that \x\ 2 — r = 1. Then 

{yeR e :Q A (x,y)<r} = {y E R e : QaM > 1}, 
{yER e :Q B (x,y)<r} = {y E R £ : Q B2 (y) > 1}, 

and the complements of these sets are given by 

D := {y E R e : Qa 2 (u) < 1}, E := {y E R e : g B2 (y) < 1}. 

Since, by our assumptions, the boundaries of D and E don't intersect transversally, we 
can apply Proposition 12.51 and find that D G E, ot E G D. Assume, e.g., that the first 
inclusion holds. Scaling in y and taking complements, we then see that 

{y E R £ : QbM >s}G {y E R £ : QaM > s} for every s > 0, 

hence 

Qb(x,v) <r => Q A (x,y) < r, 
whenever \x\ 2 > r. This implies T r B G T r A . 
2. Case: t x = 1 and r = 0. 

Then there are vectors £i, £ 2 ER l \ {0} such that A 2 = <8> and B 2 = ^ 2 ® £ 2 , i>e., 

Q^(x) := |x| 2 -(6-l/) 2 , 

g»(ar) := ^+B#i(i + %)-(6-j) 2 , 

and f2 = M £ \ (£f U ^) is non-connected. 

(a) Assume first that ^ ^ 0. Then Q consists of four connected components, on each 
of which the sign of £i • y and £2 • V is constant. Let V be one if these components such 
that V contains a point in Arguing as in Case 1, we can then conclude that V G Q±, 
so that the alternative (|2.15|) holds for every y E V. In particular, if y E V is sufficiently 
close to the hyperplane then \Dq\ < \Eq\, hence D\ G Eq. Here, \M\ denotes the 
Lebesgue volume of a Lebesgue measurable subset M of R k . Similarly, if y' is sufficiently 
close to the hyperplane then Eq G Dq . Connecting y and y' by some continuous path 
7 in V, we can thus conclude as in Case 1 that there is some 77 E V such that Eq = Dq. 
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As in Case 1, this implies that -B3 = and Bi = Ax, without loss of generality. Moreover, 
by (|2.21j) . since /3i = 1, then B 2 r\ = A 2 rj, hence (£1 • 7/)£i = (£ 2 • v)£,2- This contradicts our 
assumption that £1 A £2 7^ 0, and thus this case cannot arize. 

(b) Assume finally that £1 A £2 = 0. By a linear change of coordinates in M. , we may 
then assume that £1 • y — y\ and £2 • V — ayi, for some a > 0, so that Q consists of the 
two connected components Q± := {y G WL e : ±yi > 0}. As before, at least one of these 
components must belong to Qi. But, since D y = —Dq and Eq V = —Eq, we see that 
y G Oi if and only if — y G f2i, and thus Q = Q\. In particular, the alternative (|2.15|) holds 
for every y G Q. Following the arguments applied in Case 1, assume again that there are 
y,y' G Q such that 

D y r C El and E y r ' C Df' . 

We can then assume that y and y' ly in the same component of Q, say, e.g., Q + . 

To see this, notice that a change of sign of y does not change the volumes of D y r and 
E». Thus, if D y r C then c -D~ y = D v r would imply £7^» <Z D y r <Z E%, hence 
£-» = Dl = D~ y . This shows that Df C ^ implies Dp* C 

Then we can connect y and y' within Vt + by some continuous path 7, and thus find some 
77 G f2 + such that £g = -Do - ^ s before, this implies that B 3 = and I?i = Ai, without loss 
of generality, and moreover B 2 r] = A 2 rj, hence a — 1. But then Qa = Qb, hence T A = T° B . 

Q.E.D. 

Remark 2.7 If k — 1, the statement in Theorem 12.61 may fail to be true. Take, for 
instance, Qa(x, y) := x 2 — y 2 , Qb(x, y) := x 2 — 2exy — y 2 , e > 0, for (x, y) EM. 2 . 

Corollary 2.8 Let W C Sym (n, M.) be a non-trivial subspace of Sym (n, M), and fix A & 
W of maximal rank r in W. Let also B G W, and assume that 8Ta and 8Tb are varieties 
of dimension n — 1. 

// there is no point in 8Ta C\ 8Tb at which the boundaries 8Ta and 8Tb intersect 
transver sally, and if \\B — A\\ is sufficiently small, then either Ta C Tb or Tb C Ta, 
provided r > 3. 

Proof. Assume that A has signature (/c, £1). The case /c > 2 is then covered by Theorem 

If k < 1, then ^1 > 2, since r = + l\ > 3. The case fc = is trivial, since then 
Ta = = M n . So assume k — 1. Applying Theorem 12.61 to —A and —5, we find that 
C r_ B or r_ B C r_ j4 . Say, the first inclusion holds. Then also (T_^) C (r_g)°, 
hence 

T B = R n \(T- B )° cR n \(T A )° = T A , 
so that the statement of the corollary is true also if k — 1. 

Q.E.D. 

From Theorem 12.21 and Corollary 12.81 we immediately obtain 

Theorem 2.9 Lei A,B £ Sym (n,M) be a non-dissipative pair, and let us assume that 
maxrank {A, B} > 3. 

If A and B are linearly independent, then there exists a point z G 8Ta r\dT B at which 
the boundaries OYa and 3Y B intersect transver sally. 
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Proof. Let W := spaiiR-fA, B}, and choose a basis A, B of W such that rankA = 
maxrank {A, B}, and such that | \B — A\\ is so small that Corollary 12 . 81 applies to the pair 
A, B (notice that the boundaries dT^ and dT B have dimension n — 1, because A and B 
have signature (k, £i), with k > 1 and £i > 1, since is non-dissipative.) Observe that 

dr A ndr B = dr A ndr B . 

Assume now that there is no point z G dT^ H <9r# at which Az A Bz ^ 0. Then there 
is also no point z G dT A H dT B at which Az A i?z ^ 0, and consequently C Tg or 
r^j C T^. However, in view of Lemma 12.11 after a suitable linear change of coordinates 
we may assume that A and B have vanishing traces. Then, by Theorem 12.21 A and B are 
linearly dependent, hence so are A and B. This proves the theorem. 

Q.E.D. 

Remark 2.10 The analogous statement is false for r = 2. Take, for example, Q A (x, y) = 
x 2 -y 2 , Q B (x,y)=xy, (x,y) eR 2 . 

3 The form problem 

We begin with some auxiliary results and background information on semi-algebraic sets, 
which will be useful later. 

3.1 Auxiliary results 

Lemma 3.1 Let A, B G Sym (n, M) such that A is not semi- definite, and assume that 
dT A cdT B . 

(a) Then there is a constant c G M such that B = cA. 

(b) //rank A = 2, so that <9r^\{0} is not connected, then let us choose linear coordinates 
x = (xi, . . . ,x n ) so that Qa = ax\X2, with a G M \ {0}. If Qb vanishes on one of 
the components of dT A \ {0}, say Qb(x) = 0, if x\ = 0, then there is some feel" 
such that Qb{x) = {b ■ x)x\. 

Proof, (a) After applying a suitable linear change of coordinates, we may assume that 
we can split coordinates x = (u, v, w) G IR fc x IR £ x IR m = M. n such that 

Q A (x) = \u\ 2 - H 2 =: Q Al (u,v) . 

Setting y := (u,v) G M. k+£ , we can then write Qb as 

Qb (y,w) = Q Bl (y) + (B 2 y) -w + Qb 3 (w), 

with Bi G Sym (k + £, M), B 3 G sgn (m, R) and B 2 a real m x (A; + £)-matrix. Let y G dY ' Al . 
Then QB{y,w) = for all w G M m , hence 

(3.1) £? 3 = 0, B 2 y = and Q Bl (y) = . 

Since, by our assumptions, k > 1, £ > 1, dT Al spans M. k+e . To see this, choose unit vectors 
ei,...,efc G M fc spanning M fc and unit vectors fi, ■ ■ ■ , fg G M £ spanning IR £ . Then the 
vectors ± fj ly in dT Al and spanIR fc+£ . Then (13. 1|) implies B 2 = 0, so that 

Qb{v,w) = QbAv) ■ 
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We may thus reduce ourselves to the case m = 0. Let us then write (with new matrices 

Qb{u, v) = Q Bl (u) + Q B2 (v) + 2(B 3 v) ■ u . 
For (u',v') G S^ 1 x S^ 1 and s,t G R, we have 

q(s,t) := Q B {su',tv') = s 2 Q Bl (u') +t 2 Q B , 2 (v') + 2st(B 3 v') ■ u' 
= as 2 + f3t 2 + 2-fst. 

By our assumptions, q(s,t) = 0, if \s\ = \t\. In particular, q(t,t) = q(t, —t) = 0, so that 
7 = and a + (3 = 0. Thus 

(B 3 v') ■ u' = for all v! G S k '\ v' G S^ 1 , 

so that B 3 = 0. Moreover, 

Q Bl = -g B2 (v r ) for all u' G S^ 1 , v' G S^ 1 . 

Scaling, this implies 

QbA u ) = ~\ u \ 2 Qb 2 W) for a11 u eR k , 

hence B x = dk, for some cGi Then Qb 2 ( v ') — ~ c -, hence Qb 2 ( v ) — —c\v\ 2 for all v G 
so that B 2 = —cIi and Qb = cQa- 

(b) If Q B (x) = for x\ = 0, then put s := Xi, v := (x 2 , . . . , x n ), and write 

Qb{x) = Qb 1 (v) + s/3 ■ v + 7s 2 . 
Since Qb(v, s) = 0, if s — 0, we have Bi = 0, so that Qb = s(/3 ■ v + 7s). 

Q.E.D. 

If V and W are finite dimensional IK— vector spaces, we shall denote by L(V, W) the 
space of all linear mappings from V to W. If V — M. k and W = M. e are Euclidean spaces, 
we shall often identify a linear mapping T G L(V, W 7 ) with the corresponding n x k— 
matrix in M nxk {K) with respect to the canonical bases of these spaces, without further 
mentioning. 

Lemma 3.2 Let A G L(IR™,IR n ) and T G L(W n ~ m , M. n ) be linear mappings, and assume 
that T is injective. Then 

rank ( *TAT) > rank A - 2m. 
Proof. It suffices to prove that 

(3.2) dim(ker *T l AT) < dim(ker *A) + m, 

for then rank ( ^TAT) = n — m — dim(ker *T l A T) > n — dim(ker t A) — 2m = rank A — 2m. 
Write B :— t A. Since T is injective, f!3.2|) will follow from 

(3.3) dim(ker *TB) < dim(ker B) + m. 

Let K := ker *T. Then y G ker 'TB if and only if By G K, i.e., ker 'TB = B~ l (K), where 
clearly dim B~ l (K) < dim(ker B) + dim if = dim(ker B) + m. This gives ()3.3jh 

Q.E.D. 
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Lemma 3.3 Let A, B G L{R n , IR n ) be linear symmetric mappings, i.e., l A = A and 
l B = B, and let I be a non-empty open interval in R and E : / — » L(K n_m ,R n ) a 
differentiate mapping such that E(t) is infective for every t £ I. Assume that 

(3.4) l E(t)BE(t) = n(t) l E(t)AE(t) for every t G I, 

where /i : I — > R is a differentiable mapping. Then \x is constant, provided rank A > 4m. 
Proof. Let r := rank A, and put 

A(t) := *E(f)AE(t), B(t) := E(t)BE(t), t E I. 

Then 

B{t) = fi(t)A(t) for all tel. 

Assume that \x is non-constant. Then there is some point t & I such that /i(t ) := 
^(to) 7^ 0. Translating coordinates in M, if necessary, we may assume that to = 0. 
Moreover, replacing then B by B — fi(Q)A, we may assume that /x(0) = 0, hence B(0) = 0. 
Then 

5(0) = mo)A(o) , 

hence, by Lemma f3.2t 

(3.5) rank B(0) = rank A(0) > r - 2m . 
On the other hand, 

B(0) = *E(0)BE(0) + *(^(0)5E(0)) , 

and 

= B(0) = *E(0)££(0) . 

Since £"(0) is injective, this implies *J5(0).B|y(o) = 0, where V(0) denotes the (n — m) — 
dimensional range of E(0), so that rank *E(0)5 < m, hence 

rank *E(0)BE(0) < m . 

The same estimate holds for '( *E(0)5£'(0)), and thus 

rank 5(0) < 2m. 

Together with (|3.5J) . this yields r < 4m. 

Q.E.D. 

In the sequel, we shall consider three matrices A, B,C G Sym(n, R), and shall work 
under the following assumptions, unless stated explicitly otherwise: 

Standing Assumptions 3.4 A and B are linearly independent and form a non-dissipative 
pair, and r := maxrank {A, B} > 3. Moreover, C statisfies the following property: 

(HI) Qc vanishes on {Qa = 0} H {Qb = 0} . 

Unless stated otherwise, we also assume that rank A = r, and that \ \B — A\\ is sufficiently 
small. 
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We shall be mostly interested in the situation where M. n = M. is the canonical sym- 
plectic vector space, and Qc is the Poisson bracket 



(H2) 



Qc = {Qa,Qb} 



of Qa and Qb- Our aim will be to prove that, under these assumptions, 



(3.6) 



C = aA + (3B 



for some a, (3 G R, provided r is sufficiently large. 

Let us first have a closer look at the structure of the sets {Qa = 0} = <9r^, {Qb — 
0} = 8Tb and their intersection 



As a reference for the following results, we recommend pQ. 

The sets 8Ta,0Tb and V are real-algebraic, hence semi-algebraic, so that they admit 
a finite stratification into connected, locally closed subsets which are real-analytic subma- 
nifolds, each of which is a semi- algebraic set. Recall that the dimension of a semi-algebraic 
set is the maximal dimension of its strata. 

If we assume that B is so close to A that also rank B = r, then clearly dT a and 8Tb 
are of dimension n — 1 , so that dim V < n — 1 . 

Theorem 12.91 implies that there exist points z G V, at which the boundaries of 8Ta 
and 8Tb intersect transversally. 

Denote by Af C V the set of all those points z in V. Then Af is a non-empty analytic 
submanifold of codimension 2 in IR n , and Az, Bz span the normal space N z Af to M at 
every point z G N . 

In particular, M decomposes into a finite number of strata of dimension n — 2 contained 
in V, so that dim V > n — 2. It is easy to see that dim V = n — 1 is not possible. 

For, if dim V = n — 1, then there exist a non-empty open neighborhood U of some 
point z G V in IR n such that 



If k > 2 and £i > 2, then (91^ \ {0} is connected, and since Qb vanishes on dT a \ {0} 
within U, it vanishes on the whole of 8Ta, by analyticity. And, if, e.g., k — 1, then 
the semi-cones dT± := {x : Q A {x) = Oand ± x x > 0} are connected, and a similar 
argument as before shows that Qb vanishes on at least one of the sets dT\ or dT^. Since 
Qb(—x) = Qb(x), then again Qb vanishes on dT a- 

Reversing the roles of A and B, we see that dT a = 9T B - But then Af = 0, a 
contradiction. We thus have 

(3.7) dimV = n-2. 



V := 8T A n <9r 



<9r A n c/ = <9r B n u = dT A ndT B nu. 



However, we may apply a linear change of coordinates so that 



Qa(x) — (x\ + • • • + x\) + • • • + Xg-J ■ 
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3.2 The case where no stratum of J\f spans M. n . 
We first rule out the possibility that no component of Af spans M. n . 

Theorem 3.5 Let A,B,C G Sym (n, M.) satisfy our Standing Assumptions Assume 
that maxrank {A, B} > 7, that W 1 = M? d is symplectic, that Qc satisfies Hormander's 
bracket condition (H2) and that the joint radical TZa,b '■= ker A n ker B of Qa and Qb is 
either trivial, i.e., ker A D ker B = {0}, or a symplectic subspace of MP. 

Suppose also that no connected component of Af spans W 1 . Then C is a linear combi- 
nation of A and B. 

Proof. Without loss of generality, we may assume that Ha,b is trivial. For, if TZa,b is 
a symplectic subspace, then we can choose a complementary symplectic subspace of lR n , 
and reduce everything to this space. 

Let then Xa{x) := 2JAx, X B (x) := 2JBx denote the Hamiltonian vector fields asso- 
ciated to Qa and Qb- Since 

X a Qb = {Qa,Qb} = on Af, 

the field Xa is tangential to dT B at every point of V C Af, and trivially the same holds 
for 8Ta, so that by symmetry in A and B, 

X A (x),X B (x) e T x Af for all x G Af . 

Assume now that no connected component of Af spans M n . 
Let Ao be any component of Af. Then 

(3.8) A/" C u x 

for some unit vector v G W 1 . In particular, Xa{x),Xb{x) G v , hence 

v G span R 1X^(0;), X B (x)} ± for all x G Ao . 

Here, _L denotes the orthogonal with respect to the canonical Euclidean inner product on 
]R n . Then Ju G spanM.{Ax, Bx} x , hence 

(3.9) Ju G T x Nq for every x G Ao . 
Let 

V := (span M {z/, Jv}) x . 

Then V is a J-invariant, symplectic subspace of W 1 , and we can choose ortho normal 
coordinates x\, . . . , x n , so that 

v = e n _i, Ju = e n , 

where e\, . . . , e n denotes the associated basis, and where e\, e^, . . . e^d-i, £2, ■ ■ ■ , &2d forms 
a symplectic basis of lR n . 

Representing Ao locally as a graph, we see that (|3.9|) implies that Ao is locally a 
cylinder with axis e n , over a basis AA C M n_1 ; by analyticity of Qa and Qb, we see that 
this holds globally: 

(3.10) Af = A4xR, 
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where 

M:={yE R n - 1 : (y, 0) G M } C R™ -1 . 

Notice also that, since J\f C v L = e^_ 1: we have indeed that M. can be considered as an 
(n — 3)- dimensional submanifold 



(3.11) 



M c {y e 



r>n— 1 



y n _ x = 0} =: H 



of the (n — 2)-dimensional hyperplane H in M. n 1 , which can naturally by identified with 
V = H x {0}. 

Splitting coordinates x = (y,s), y G IR n_1 , s G R, we can write A in the form 



* 1 " 2 ) , with a 2 G 

Ct 2 a 3 



»n- 1 



~ e n , a 3 G 



Then 



s) = Q^j/) + 2s(a 2 • y) + a 3 s 2 . 
Since Qa^, s) = for all s G R, if y G .M, we find that 

(3.12) a 3 = 0, a 2 • y = and Q Al {v) = for all y G .M . 

In particular, a 2 G .M -1 . 
Case 1. span ffi .M = H. 

Then a 2 G if -1 , hence a 2 = re„_i for some r G R, so that 

/ 



A = 



A, 



V 



0---0r 



\ 



Interchanging the roles of A and 5, we see that B is of the same form 





( 


\ 


B = 


B 1 













\ 0- • -0(7 





with a" G R. If both r and cr were zero, then e n would be in the joint radical 1Za,b of Qa 
and Qb, which is assumed to be trivial. Thus, at least one of these numbers is non-zero, 
and forming suitable new linear combinations of A and B (dropping the assumption that 
\\B — A\\ be small), we may assume without loss of generality that 





( 


o\ 




A = 


A, 





, B 






1 






\ 0---01 
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so that 

(3.13) Qa(v,s) = QaAv) + 2sy n _i, 

Qb(v,s) = QbAv)- 

Since A and I? are linearly independent, we have rank B\ > 1. Moreover, rank I?i = 1 is 
not possible, since then would be semi-definite. Therefore, rank B\ > 2. 

Case 1(a). rank B\ = 2. 

Since B\ is not semi-definite, we can then find linearly independent vectors 771, 772 G M n_1 
such that 

Qsi(y) = (vi ■ y)(v2 ■ y) • 

Moreover, by (|3.13jl . 

vn{(?/,s)e M"- 1 x r : y n _! = 0} = (<9r Al n <9r Bl nff) xi, 

so that .M must be an (n — 3)-dimensional stratum of Pi 8Tb 1 D H of codimension 
1 in i/. In particular, M. is contained in either 77^, or in 77^. Assume, e.g., that 

M C 77^- . 

If ?7i A e n _i 7^ 0, then lies in the subspace K := {y G IR™ -1 : ?/„_! = 0,771 • 7/ = 0} 
of codimension 1 of H, hence is an open subset of K. Since Qa 1 vanishes on A4, we see 
that Qax vanishes on K. Write 



At 



l a 2 a' 3 



with respect to the splitting of coordinates y = (y',y n -i). Then Qa'Av') — whenever 
77^ • y' := 771 • (y', 0) = 0. Lemma I3~T1 (b) then implies that Qa'^u') = (v[ ' 2/0 (^2 ' 2/); f° r 
some 772 G IR n ~ 2 . In particular, rank A[ < 2. 

On the other hand, if we apply Lemma 13.21 where T : IR"~ 2 — > M. n is the inclusion 
mapping y' — > (7/, 0) G IR™~ 2 x ]R 2 = R n , then we see that rank > rank A — 4 = r — 4. 
Thus r < 6, in contrast to our assumptions. 

Let us therefore assume that 77! A e„_i = 0, say, without loss of generality, 771 = t n ~\- 
Then 

Qb(v,s) = Un-xti • y) , 

for some 77 G IR™ -1 . Write 77 = ae n -\ + v, with v G span^ei, . . . , e„_2} = V. Then v ^ 0, 
since Q/3 is not semi-definite, so that we can consider f as a member of a new symplectic 
basis of V. Replacing {ei, . . . , e n -2\ by this new basis, we may then assume without loss 
of generality that v — ei, i.e., 

Qb(v, s) = cry 2 ^ + yxyn-i = y n -i{vVn-i + Vi), 
Qa(v, s) = Qax (y) + 2sy n -i , 

and that the Poisson bracket of y\ and y 2 satisfies {1/1,1/2} = 1- Since {s, y n -i} = 1? we 
then obtain 

d 

Qc = {Qa,Qb} = 2y n ^ 1 (2ay n -i +y 1 ) - y n - X —^ QaAV)- 
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Thus, 

Qc = Vn-i(i ■ y) , 

for some 7 G IR n_1 . 

Assume that y n -\ 7^ 0, and oy n -\ + V\ = 0. Then Qb{v,s) = for every s G R. 
Moreover, we can choose s = s(y) such that Qa{Vi s) = 0. Then, by (JHTJ), Qc{y, s ) — 0. 
Thus, cry„-i + 2/1 = always implies 7 • y = 0, so that 7 = c(ei + <re n _i), for some ceR, 
hence Qc(y) — c 2/n-i(yi + vyn-i) = c Qb(v)- Thus C lies in the span of A and B. 

Case 1(b). rank B x > 3. 

Since i?i is not semi-definite, every connected component of 8Tb 1 \R then spans M n_1 , 
where R denotes the radical of Qb 1 - This can be seen by a similar argument as in the 
proof of Lemma f3 .11 Let V be such a component of 8Yb 1 \ R, and consider 

M) := {(y,s(y)) eR n :yeV, y n . x > and s(y) = -\Q M {y)/y n -x}. 

Then Ao is a connected submanifold of dimension n — 2 contained in V, hence contained 
in a stratum Af of maximal dimension. By our assumption, A/o is also contained in a 
hyperplane. Since V spans, we can then find some vector 77 e ]R n_1 such that 

s(y) +vy = for all y e V + := {y G V : y n ^ > 0}, 

hence 

Qai (y) = tyn-iiv ■ y) for every y G R" -1 . 

Put 

Qdx (y) := Qax (y) - 2y n _ 1 (7 ? • y), y G M^ 1 . 

Then vanishes on P + , hence, by analyticity, also on the stratum Q of containing 
V + , as well as on —Q. However, QU (— fl) is dense in Sr^, so that Q^i vanishes on ^r^. 
By Lemma f3. II (a), there is thus a constant c G K such that 

Qz?i = cQb 1 ■ 

Replacing A by \{A — cB) (possibly dropping the assumption that rank A = r), we see 
that 

(3.14) Qa(v, s) = y n -i(v y) + s y n -i = y n -i(v y + s), 

Qb(v,s) = QbAv)- 

Similarly as in the previous case, we can now choose symplectic coordinates in V such 
that 

vy = oVn-\ +PVi, a,peR, 

so that 

Qa(v, s) = y n -\{py\ + vy n -i + s). 

Then 

d d 
Qc = {Qa,Qb} = 2/n-io QbxKV) ~ Pyn-ia—QBtW 

oy-n-x oy2 

= y n -i(i-y), 
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for some 7 G IR n 1 . We may assume 7 7^ 0. 

Let then y G dT Bl such that y n ~\ 7^ 0. Then we can choose s = s(y), such that 
(y, s) G V, hence, by flHH), Vn-x{l ■ y) = 0. 

This shows that {?/ G M n_1 : Qb 1 (v) = 0, y n _i 7^ 0} lies in the hyperplane 7- 1 -, so that 
9rs 1 is contained in the union of two hyperplanes. This is, however, not possible, since 
rank B\ > 3, so that the strata of dimension n — 1 in dBi are not flat. 

Case 2. span M ^ H. 

Let := span^A^. Since A4 has codimension 1 in H, we see that W has codimension 
1 too, so that M. is an open subset of W. And, Qa, Qb vanish on Ao = M. x K, hence 
also on 1^ x E, so that M. = W and A/o = x K. Since A/o is a linear subspace of 
codimension 2 in IR ra , we may introduce new orthonormal coordinates x\, . . . ,x n in IR n 
such that 

Ao = {x G lR n : x n _i = x n = 0} . 

Splitting coordinates x = (z, y), with z := (xi, . . . , x n _2) £ M n ~ 2 , y := (x n _i, x n ) G M 2 , 
we can therefore write 



with respect to these blocks of coordinates. Notice that rank A2 < 2. 

Then, by Lemma f3. 21 r = rank A < 4, in contradiction to our assumptions. So, under 
the hypotheses of Theorem 13.51 this case cannot arize. 



Remarks 3.6 (a) The statement of Theorem 13.51 fails to be true, if r to small, e.g., if 
r = 4. 

Consider, e.g., the following counterexample in M 4 = M. 2 x R 2 , with coordinates 
(x,y), x = (xi,x 2 ), y = (2/1, 2/2), from 0, Corollary 1.4: 



The corresponding matrices A and B are non-degenerate. Moreover, putting £ := \x\, X2), 



Qa = £-(Jv), Qb = -£-v, 

and thus {Q A = 0} n {Qs = 0} = {(£,77) G M 2 x R 2 : £ = or 77 = 0}. Therefore, none 
of the connected components of M span IR 4 . However, 



so that A, B and C do satisfy our standing assumptions and are linearly independent. 

(b) Also the condition on the joint radical of Qa and is important, as the following 
example shows: 




Q.E.D. 



Qa(x, y) ■= x ± y 2 + x 2 yi, Qb(x, y) := x x y x - x 2 y 2 ■ 




Qc = {Qa, Qb} = 2(xiy 2 - xzVi) , 
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Let ei, . . . , e<i, fi, ■ ■ ■ , fd be a canonical symplectic basis of IR , d > 2, with associated 
coordinates (x, y) = (xi, . . . , x^, yi, . . . , yd), and put 

(3.15) Qa(x,v) ■= x{-(yl + ■■■ + Vd-i + x l + --- + x d-i)i 
Q B (x,y) := x x x d . 

Then A, B form a non-dissipative pair, since, after a suitable scaling in x±, we may assume 
that the matrices corresponding to A and B have vanishing traces. Moreover, 

Qc ■= {Qa,Qb} = -2yix d , 

so that and A, B and C are linearly independent. Then rank A = 2(d—l), and 1Za,b — ^-fd 
is isotropic with respect to the symplectic form on M. 2d . Moreover, A, B and C are linearly 
independent. Nevertheless, Qc vanishes on {Qa = 0} fi {Qb = 0} . 

(c) The condition that Qc '■= {Qa,Qb} cannot be dropped either, as the next example 
demonstrates: Let 

(3.16) Qa(x, V ) := x\- (y 2 1 + --- + y 2 d + x 2 2 + --- + x 2 d _ 1 ), 

Q B {x,y) ■= x x x d . 

Then again A,B form a non-dissipative pair, as can be seen as before, and TZa,b = {0}. 
Let 

Qc ■= VjX d , 

for any j = 1, . . . , d. Then A, B and C are linearly independent, and rank {A, B} = 2d. 
Nevertheless, Qc vanishes on V = {Qa = 0} fl {Qb = 0} = IRe^ U {(x, y) : Xd = 
and QA{x,y) = 0}. Notice that here the strata of V of maximal dimension ly in the 
hyperplane {x^ = 0}, but there is a lower dimensional stratum not lying in this hyperplane. 

If we slightly modify Qa, by putting 

Qa(x, y) := x\ - (y\ + h yl + x\ H + x 2 d ), 

the situation remains the same, only that now V lies completely in the hyperplane {xd = 
0}. 

In all these examples, minrank {A, B} = 2. We shall prove later in Lemma 13. 131 that 
this is necessarily so, if no stratum of M spans W 1 . 

3.3 The case where at least one stratum of M spans W 1 . 

Let us now go back to our standing assumptions, not requiring that W 1 is symplectic and 
C satisfies (H2). However, let us assume that 

(3.17) span R Ao = W 1 , 
for some connected component A/o of N '. 
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Notice that then, if U is a non-empty open subset of Af , also U spans IR n , by 
analyticity and connectivity of AV For xq G Af, we denote by 

V Xo := T Xo Af = (span M {Ac , Bxq})^ 

the tangent space to N aX x$. 

Moreover, if D G Sym (n, M.), then 

<5d,z : = Qd\v xo 

denotes the restriction of the quadratic form Qd,x to V XQ . Similarly, if xo,x± G Af, we 
put 

V( X0 , X1 ) ■■= v xo n v Xl , 

Qd,x , Xi ■= Qd\v {xq , xi) ■ 

Lemma 3.7 Given x G Af, there exist a(xo), (3(xq) G IR such that 

(3.18) Qc,x = oi(xq)Qa,x q + P(x )Q B ,x 
and 

(3.19) Cx = a(x )Ax + (3(x Q )Bx . 

The a(xo), /3(xo) may not be unique, but can locally on Af be chosen as real-analytic 
functions of x . 

Proof. Fix x G Af. Since VQa(xq) = 2Aro and VQb(xq) = 2Bxo are linearly indepen- 
dent, possibly after relabeling the coordinates of R n , we may assume that 

4>(x) := (xi, . . . , x n _ 2 , Qa(x), Qb(x)) =: (x', y) 

is a local analytic diffeomorphism near xq. In the new coordinates (x',y), the point xq 
corresponds to (x' Q , 0), and Af to {y = 0}. Let / := / o if / is a function defined near 
Xq. Since Qc vanishes on Af, a Taylor-expansion of Qc near (x' , 0) shows that 

Qc(x', y) = yia(x', y) + y 2 f3(x', y) , 

for suitable analytic functions a, (3 defined near (x' ,0). Thus, near xq, 

(3.20) Q c (x) = a(x)Q A (x) + (3(x)Q B (x) , 

for some analytic functions a, (3 defined near xq. Taking first derivates at xq, we obtain 

Moreover applying the second derivative to ()3.20|) . we obtain 

(3.21) C = a(x )A + (3(x )B + M(x ) , 
where 

(3.22) 2M(x ) := a'(x ) <g> Ax + Ax g> a'(x ) 

+(3'(x ) ® Bx + Bx ® /3'(x ) • 
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This implies (|3.18l) . 



Q.E.D. 

For xq, x\ G A/"o, let 

N( X0)Xl ) ■= span R {Axo,Bx , Ax 1 ,Bx 1 } , 

so that V( X0 , X1 ) = Nfe 0>xi y Then dim N( X0}X1 ) > 2. If dim N( X0>X1 ) = 2 for all (x , Xt) G 
Aq 2 , then V Xl = V Xo for all xo, x\ G A/o, so that A/o would not span IR n . So, if we put 

m : = max dim N( X0>X1 ) , 

then m G {3, 4}. 

Let us call a subset {7 of an open domain Q C lR m a generic set or set of generic 
points in f2, if there exists a non-trivial real analytic function / : Q — > R such that 
J7 = {a; G : /(a;) ^ 0}. A property will hold for generic points in Q, if it holds for all 
points of a generic subset. 

Notice that a generic set in Q is open and dense in Q. Clearly, the intersection of a 
finite number of generic sets in Q is again generic. 

Moreover, if m = k + £, and if U is a generic subset of VI C R k x M £ , then let fi 1 C 
]R fe , f2 2 C M. e be any domains such that fi 1 x f2 2 is compact in f2. Then U\ := {x G fi 1 : £4 
is generic in Q 2 } is generic in O 1 . Here U x denotes the x-section 

U x := {yen 2 :(x,y)eU} 

of U. Indeed, if U = {(x,y) G : f{x,y) ^ 0}, then Vli\U\ = {x E Vl 1 : D«f(x,0) = 
for all a G F^}. Thus, if we put 

g(x) := ^e QJ D°/(x,0) 2 , 

for a suitable family at coefficients e a > tending to sufficiently fast as \a\ — > oo , then 
(7 is a non-trivial real analytic function on Q 1 , and 

U x = {x G VI 1 : ^ 0} . 

Analogous definitions and results apply for open domains Q in a real analytic manifold. 

If m = 3, and if dim Nr XOtXl ) = 3, then three of the vectors Axq, Bxq, Ax\, Bx\ are 
linearly independent, say, e.g., Axq A Bxq A Aci 7^ 0. But then this holds for generic 
pairs (xo,xi) G A/" 2 , by analyticity and connectivity of A/o, hence dim V^^) = n — 3, for 
generic (xo,xi). A similar argument applies to the case m = 4. Therefore, the set 

g := {(x Q , £1) G A/" 2 : dim V^^) = n - m} 

is generic in Aq 2 , and dim V[ Xo ^ > n — m for all (xo, X\) G A/" 2 . 

Proposition 3.8 Assume that A, B and C satisfy our standing assumptions, and that 
(I3.17j) holds. If Qa,xq, Xi and Qb,x , Xi are linearly independent for some (xo,xi) G Q, then 
C lies in the linear span of A and B. 
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Proof. Choose connected open neighborhoods Uj of Xj, j = 0, 1, in Ao such that U xUi C 
Q. For (y, z) G U x U x , we have, by (jSZHJ), 

Qc, y ,z = a(y)Q A , y ,z + !3{y)QB,y,z , 

as well as 

Qc\ y ,z = a(z)Q A , y , z + /3(z)Q B>y ,z ■ 

Moreover, Qa,x , Xi A Qb,x , Xi 7^ 0. Shrinking U and U\, if necessary, we may therefore 
assume that also Qa,v,z A Qb, v ,z ^ for every (y, z) E Uo x Ui, so that 

a{y) = a(z), (3(y) = j3{z) for all (y, z)eU xU 1 . 

Putting «o := Oi(x\), Po '■= P(xi), we see that 

(3.23) a(y) = a , P(y) = Po for all y G U . 

But, since Uo is a non-empty open subset of No, and since A/o spans M n , then also Uo 
spans 1R", by analyticity and connectivity of No. ()3.23j) and (j3.19j) imply 

Cy = a Ay + (3 By for all y G U , 

and since U spans, we obtain 

C = a A + p B • 

Q.E.D. 

There remains the case where 

(3.24) Qa,x ,x! A Qb,x ,xx = for all (x , x{) G Q . 

Lemma 3.9 Let A and B satisfy our standing assumptions, and that there exists a non- 
empty domain U C Q such that Qa,x, v and Qb,x, v are linearly dependent for every (x,y) G 
U. Then there exists a non-trivial linear combination D = aA + (3B of A and B such 
that 

Qd,x, v = for every (x, y) G U , 

provided r > 17. 

Proof. Since N( XjV ^ = span^ {Ax, Bx, Ay, By} varies analytically in (x, y) G Q, shrinking 
U , if necessary, we may assume that there is an orthonormal basis (constructable, e.g., 
by the Gram-Schmidt method) 

e 1 (x,y),...,e n _ m (x,y), (x,y)eU, 

of V( Xy y), varying analytically in (x,y). Put 

E(x, y) := ( ei (x, y)),..., e n _ m (x, y) G M n ^ n ~ m \R) , 

and 

A(x,y) := t E(x,y)AE(x,y), 

B(x,y) := t E(x,y)AE(x,y), (x,y)eU. 
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Then A(x,y) and B(x,y) are real (n — m) x (n — m)-matrices, representing the forms 
Qa\v, xv) an d QB\v (xy) with respect to the above basis of V( x>y ). By Lemma l3~2) we have 

(3.25) rank A(x, y) > r — 2m 

If r > 9, we have in particular that A(x, y) ^ 0. Since Qa,x, v A Qb,x, v = 0, there exists 
thus a unique a(x, y) el such that 

(3.26) S(ar,j/)=a(x,y)A(o;,2/), (x,y)eU. 

Since a(x,y) is unique, it depends analytically on (x,y) G £7. 

Assume a is non-constant on U . Then we can choose a differentiable curve 7 : 7 — ► U, 
such that /1 := a o 7 : 7 — > IR is non-constant. Put 77 (i) := 77 (7 (t)), and 

A(t):=A( T (t)), 77(t) := 77( 7 (i)), t E I. 

Then 

77 (t) 7777(f) = /i(t) '77 (t) 7177 (i) for every ie]-l,l[, 

so that, by Lemma [3. 31 r < 4m < 16, contradicting our assumptions. 

Consequently, a is constant, i.e., a = ao, for some ao G K. Putting D := B — aoA, 
we find that 

Q D,x,y , 

first, for every (x,y) in our shrinked domain U G Q, but then also for all (x,y) in our 
original domain U . 

Q.E.D. 

Lemma 3.10 Assume that M^spansW 1 , and that r > 12, an7 /e£ JJ\ be a non-empty 
open subset of Mq. We also assume 1)3.24)1 . 

Then, for generic Xq G £7i ; i/iere exists an X\ G C7i 7 suc/i t/iat for every sufficiently 
small neighborhood JJ\ of xq and U2 of x\ in Ux, the following hold: 

U := Ux x U2 C Q , and the union of the spaces V(x >y ), (x,y) G U, spans M 71 . 

Proof. Without loss of generality, we may assume that \\B — A\ \ is so small that rank 77 = 
rank A = r. 

For generic x G Ux, there is some x\ G U\ such that (x ,Xx) G Q. Given such a pair 
(x ,xx), choose open neighborhoods Ux of x and U 2 of x\ in Ux such that U GQ. 

Assume now that LK^Oe,?/) '■ ( x iU) ^ U} does not span W 1 . Then there is a unit vector 
v G 1R™ such that z/ J_ V^^), i.e., v G N( XjV ), for all (x, y) G £7. 

We can exclude that v A Ax A 77x = for every x E Ux- For then v G span {TLx, 7? a:}, 
hence V x C z/^ for every x E Ux- But then 14 C for all x E Ao, by analyticity and 
connectivity of Ao. This implies z ■ v — const, for all z E Ao, and since G Ao, we have 
z • v = 0. Thus Ao C v L , in contradiction to our assumptions. 

Thus, v A Ax A Bx 7^ for generic x E Ux- Choose x' E Ux and an open neighborhood 
U[ of x' Q in Ux, such that 

(3.27) v A Ax A Bx ^ for every x G £7{ , 
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and put U' := U' x X U 2 C U. 

Assume now first that m — 3, and that, e.g., 

iV^j,) = span M {Ax, Bx,Ay} , 

first, for (x, y) = (xq, X\), and then for every (x, y) E Z7' (shrinking U[ and L^, if necessary). 
Since v E N( XiV ), we have 1/ A Ax A Bx A Ay = 0, hence, by (|3.27|) . 

Ay G spans {v, Ax, Bx} for all (x, y) E U' . 

Fixing x E U[, we see that Ay lies in the 3-dimensional subspace span R {v, Ax, Bx}, for all y E 
U 2 , so that {Ay : y G L^} does not span M. n . But then f/ 2 does not span either, for oth- 
erwise, span {Ay : y E U2} = range A would have dimension r, hence r < 3. The case 
where Nr x ,y) — s P an R {Ax, Bx, By} can be treated in the same way. 

There remains the case m = 4. Then 

Ax A Bx A Ay A By 7^ , 

and 

v E spans {Ax, Bx, Ay, By} for all (x, y) E U[ x U 2 ■ 
By ()3.27p . this implies that there exists (a(x, y), (3(x, y)) E M. 2 \ {0}, such that 

(3.28) a(x, y)Ay + (3{x, y)By E span K {v, Ax, Bx} for all (x, y) E U[ x U 2 ■ 

Moreover, since r > 9, the proof of Lemma 1331 ( see 1)3.26]) ) shows that we may assume 
that 

Qb ,XQ,Xl 1 

for some ao G R. Put D := B — a$A. Then D / 0, and Qd,x ,xi = 0. By Lemma 13.21 
then implies that rank/} < 2m = 8. 

Fix x in 1)3.28]) . and put W x := span^ {v, Ax, Bx}. Then 

a(x, y)Ay + /?(x, y).By G W x for all y G U 2 . 

Writing B = a A + D, we have, by (l3~2~%|) . 

(a(x, y) + a /3(x, y))Ay G + range D. 

If a(x, y) + aco/3(x,y) 7^ for some y G f/2, then this implies r = rank A < 3 + 
rank D < 11, so that this case cannot occur. Assume therefore that a(x,y) + a j3(x,y) = 
for all (x,y) G U[ x [7 2 . Then, by (13~2%|) . 

/9(x, y)(— q; A + B)y E W x for all y E U 2 . 

But, since rank A = r > 12, ()3.28)) implies that (3{x,y) cannot vanish identically on C/2, 
for generic x G Z7(, so that 

Dy = (-a A + 5)y G W x , 
for generic (x, y) G U{ x £/ 2 . Thus, since U 2 spans M n , 

ranged C Q W x —. W, 
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for some non-empty open subset V\ in U[. In particular, rank D < 3. 
But, if W x = W y for all x, y G V\, then, for fixed x, 

Ay G W x for every y G V\ , 

hence ranged C W x , since V\ spans lR n , a contradiction. Therefore, dimW < 2, hence 
rank D < 2. Since .D cannot be semi-definite, we thus have rank/} = 2 . Then, there are 
£, 77 G IR n \ {0} such that Qd(x) = (£ • a?) (77 • x). But this implies 

Ao C orA^o Cr^, 

so that Ao does not span M. n , in contrast to our assumptions. 

Q.E.D. 

Lemma 3.11 Let A,B£ Sym(n, R) satisfy our standing assumptions. Assume fur- 
ther that there is a non-empty open subset U C Q such that the union of the spaces 
V{x,y)) { x iV) £ U, spans W 1 , and r > 12. If D G sp&n^{A, B}, then the condition 

(3.29) Q DtX>y = for all (x, y) G U 
implies D = 0. 

Proof. Assume that D satisfies ()3.29|) . but that Then spanR{^4, D} = spanK{A B}, 

so that we may assume without loss of generality that D = B. Notice, however, that we 
can then still assume that rank A = r, but no longer that also ranki? = r (deviating thus 
slightly from our standing assumptions), since B then may not be close to A. We shall 
show that these assumptions lead to the contradiction that B = 0. 

As in the proof of Lemma 13.91 we may assume that there is an orthonormal basis 
e x {x, y), . . . , e n _ m (x, y) of V {XtV) , varying analytically in (x, y) G U. 

(a) We begin with the case m = 4. Putting 

fi(x,y) := Ax, f 2 (x,y) := Bx, f 3 (x,y) := Ay, f 4 (x,y) := By, 

then 

ei(x, y),..., e n _ 4 (x, y), ft(x, y),..., U(x, y) 
is a basis of lR n . Consider the mapping F : U x IR n ~ 4 — > M. n , 

n— 4 

F(x, y, z) := ^ z jej (x, y), (x, y)eU,ze R n ~ 4 , 
3=1 

where we consider U as an analytic submanifold of lR n x W 1 of dimension 2(n — 2). 
We shall prove that there is some (xo, yo, Zo) such that 

(3.30) rank DF(x , y , z ) = n . 

This implies that F is a submersion near (x ,y ,z ), so that [\{ x ,v)&u^{x,y) contains a 
non-empty open subset Q of IR n . Since, by ()3.29|) . Qb vanishes on Q, then Q B = 0, hence 
B = 0. 
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To prove (|3.3U|) . notice that 



T( X ,y)U = V X XVy, 



and 



V x = E R n : £ ■ (Ax) = 0, £ • (flx) = 0} . 
Consider, for x, z fixed, the linear mappings 

^ := 14xK^R, 
^(£,?7) := fifay) ■ (D {x , y) F(x,y,z)(tv)), i = 1, . . . 4 . 

We claim that (|3.3U|) holds for (x , yo, ^o) := ( x , Vi z), if and only if the linear mapping 



* : = 



: V x x 14 



has rank 4. 

To this end, observe first that F(x, y, z) is linear in z, so that 

(3.31) DF{x, y, 77, C) = D {x>y) F{x, y, z)(£, 77) + y, 0- 

Noreover, if f*(x,y), . . . , f£(x, y) denotes the dual basis of N(x, y ) with respect to the basis 
fi(?,y),...,f4,(x,y), i.e., if 

and if ^(6, ^) — w i 5 i = 1, . . . , 4, then let us put 



v 



Then 



1,...,4, 



so that v - D( X! y)F(x,y,z)(£,r}) G V"^, i.e., 

n— 4 

D( x , y )F(x,y, z)(£,r)) - v = - ^ C^(x, y) , 
for some (unique) £' = (Ci> • • • > C-4) £ M™ -4 . This implies, by (I3.31|) . 

n— 4 

D( x , y , z )F(x, y, z)(£, rj , ( + (') = v + ^ (j e j( x > v) for a11 C e 

i=i 

showing that DF(x,y,z) is surjective if and only if \1/ is surjective. 
Observe next that 

fi(x,y) ■ F(x,y,z) = 0, i = l,...,4, 



jn-4 
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so that, by the product rule, 





= -[ 
= -J 


ies 






/ 


\ MZ>v) J 


V 



\B-F(x,y,z)) 










\ 



\A-F{x,y,z)) 
\B-F(x,y,z)) J 



J i\v, 



? 0. 



(f , rj)eV x xV y . 

Notice that, for (x,y) G U fixed, there is some z G IR™ -4 such that ipi 
Indeed, otherwise we would have (Av) ■ £ = for all v G V( XjV ), £ G V x , and in particular 
Qa,x,v — 0. But then rank A < 2 • 4 = 8, hence r < 8, a contradiction. This shows that 
■01 7^ and 03 ^ 0, for generic z, so that 

2 < rank ^ < 4, for generic z . 

If rank \l/ = 4 for some (x, y, z), then ()3.30|) holds, and thus again B = 0. 

So, assume rank $ < 3 for every (x, y,z) G U X IR n_4 . Then, either the first two 
rows of or the last two rows are linearly dependent, when considered as linear forms 
on V x X V y (for generic z). 

If the first two rows are linearly dependent for every (x, y, z), putting v := F(x, y, z) G 
V( x , y )i we see that there is some coefficient vector (a(x, y, v), /3(x, y, v)) G R 2 \ {0} such 
that 



(3.32) 



(a(x, y, v)Av + (3(x, y, v)Bv) ■ £ = for all £ G V x 



for every (x, y) eU,v G V( x>y ). 

Choosing £ = v G V(x,y), in view of ()3.29|) this yields 



a(x,y,v)Q A (v) 



0. 



and by Lemma l3~21 (compare also (|3.25j) ). since r > 9, Q^C^) 7^ for generic t>, so that 
a(x, y,v) = for generic v G V( XjV ), hence (3(x,y,v) ^ for generic v G V^). Thus 

(Bv) • £ = for generic v G V( X)V ) , and all ^ G . 

But then this holds for all v G V^), £ G V^, so that 

(3.33) S(V(a; )2/ )) C span r{ Ac, Bx} for all (x, y) G ?7 . 

We now distinguish two cases. 

Case a.l: rank 1 !' = 2 for every (x,y) G U and generic z. 

Then the first two lines, and also the last two lines of \I/ are always linearly dependent, 
for generic z, and the preceding discussion, in particular (|3.33|) . shows that 



B(V( X; y)) C span K {Ac, Bx} 
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and analogously also 

B(V(g. tV )) C span M { Ay, By} , 

for all (x, y) G U. Thus 

-Bty^y)) C span R {Ax, Bx} n span K {A?/, By} = {0} , 

hence B\ V(xy) = 0. Since UfxjOet/^fotO s P ans K", this implies again 5 = 0. 
There remains 

Case a.2: rank ^ = 3 for generic (x,y,z) EU x IR n ~ 4 . 

Shrinking U, if necessary, we can then assume that, e.g., the last two rows of ^ are 
linearly independent for generic (x, y, z), and consequently the first two rows are linearly 
dependent, for every (x,y,z) G U x IR n-4 . Freezing x, for generic x, and applying the same 
reasoning as before to the mapping F x : (y, z) \— > F(x, y, z) instead of (x, y, z) i— > F(x, ?/, z) 
(by setting £ = 0), we see that rank Q^) = 2 implies that D( y ^F x (x, y, z) has rank n — 2, 
for generic (?/, z). This implies that, for every x in some open set U\ C Ao, the image W x 
of _F X contains an analytic sub manifold Q x of dimension > n — 2. 

But, if ^2 is an open subset of Ao such that U\ x Uz C Z7, then 

n s C W x := {F(x, y,z) : y E U2, z G IR n_4 } = |J V^) C V x . 

yeu 2 

Since dim V x = n — 2, we see that is an open subset of V x . 

And, since ()3.33|) still holds in the present case, we see that B(Q X ) C span r{ Ax, Bx}, 
hence 

(3.34) ^(K) C span ffi {Ax, £x} for all x G Z7i . 

By Lemma f3. 10\ we can find non-empty open subsets U\, U2 in U\ such that [/' := U\ x 
C^2 C Q, and so that U(xj/)g(7' Vfav) s P ans Then, by (|3.34|) . also 

B(V y ) C span K { Ay, By} for all y E U 2 , 

so that 

C span R {Ax, Bx} fl spanft{Ay, £>y} = {0}, 

for all (x, y) G U'. Again, we arrive at the contradiction that B = 0. 
(b) Assume next that m — 3. 

Except for an exchange of the roles of x and y, there are then the following two possibilities: 

(b.l) Ax A Bx A Ay 7^ 0, for some (x, y) = (x , y Q ) G U. 

Then, after shrinking U, if necessary, the same holds for all (x, y) G U, so that 

N( x ,y) — span k{ Ax, Bx, Ay} for all (x, y) G U . 

(b.2) Ax A Sx A By 7^ 0, say, again, for all (x, y) G U (after shrinking U). Then 

N(x, y ) = span K {Ax, -Bx, By} for all (x, y) G U. 
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We begin with Case (b.l). Since the arguments are quite similar to the ones in Case (a), 
we shall content ourselves with a brief sketch, just indicating the necessary modifications. 
Choose again an orthonormal basis 

ei(x,y), . . .,e n - 3 (x,y) 

of V^y), varying analytically in (x, y) G U, and put, for (x, y, z) fixed, 



/ MM \ ( %A-F(x,y,z)) 
= Mtv) ]=-{ XB-F(x,y,z)) 

\MC,V)J \ \A-F(x,y,z)) 




Then, for (x, y) G U, 

2 < rank\P < 3, for generics. 

If rank \I/ = 3 for some (x, y, z), then the image of F contains again an open subset of 
]R n , and we conclude again that B = 0. 

So, assume that rank ^ = 2, say, for all (x, y) G U, z ^ 0. Then, the first two rows of $ 
are linearly dependent, so that ()3.33j) holds. Moreover, similarly as in Case (a. 2), for fixed 
x, the mapping F x : (y,z) i— ► F(x,y,z) contains an analytic submanifold of dimension 
> (n — 3) + 1 = n — 2 in its image, which is itself contained in V x . Since dim V x = n — 2, 
we can conclude as in Case (a. 2). 

We are left with Case (b.2). Here, 

Ay G span K { Ac, Bx, By} for all (x, y) G U . 
Let us assume that U is the direct product U = U\ X U 2 of open sets Ui,U 2 C Ao- Then 
(3.35) Ay 6 + range 5 for all y eU 2 , 

for every x EU\. 

Now, since Qs,x,y — and dim = n — 3, we see that rank B < 2 • 3 = 6, so that 

dim (N x + range B) < 8 . 
On the other hand, since C/2 spans W 1 , and since rank A = r, (|3.35|) implies that 

dim (N x + range -B) > r. 
In combination, we find that r < 8, contradicting our assumptions. 

Q.E.D. 

Combining Proposition 13.81 and Lemma f3. 91 to Lemma f3. Ill we obtain 

Theorem 3.12 Let A,B,C G Sym (n, M.) satisfy our Standing Assumptions^^ Assume 
that maxrank {A, 5} > 17, and t/iat at least one connected component of Af spans M. n . 
Then C is a linear combination of A and B. 

Applying the same type of technics, we can now obtain further information also on 
the case where no stratum of M spans. 
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Lemma 3.13 Let A,B£ Sym(n, R) form a non-dissipative pair, and assume that 
maxrank {A, B} > 9. Then the following are equivalent: 

(i) There is a connected component of M which does not span W 1 . 

(ii) No connected component of M spans W 1 . 
(Hi) minrank {A, B} = 2. 

Proof, (iii) ==>- (ii). If minrank {A, B} = 2, then we may assume without loss of 
generality that ranki? = 2. This means that there are linearly independent vectors £,rj G 
]R n such that Qb{x) = (£ • x) (rj ■ x). But then clearly every component of Af lies in one of 
the subspaces or r\ _L . 
(iii) ==>- (ii) is trivial. 

(i) ==>- (iii). Assume that there is a connected component A/q of M which lies in a 
subspace u^, where v is a unit vector. Without loss of generality, we may also assume 
that rank A = maxrank {A, B} > 9. Let xq G Mq. Arguing as in the proof of Lemma f3. 71 
we see that 

(3.36) v ■ x = oi{x)Qa{x) + P(x)Qb(x), 

for all x in a sufficiently small neighborhood of Xq. Here, a and /3 are analytic functions 
near xq. Applying the second derivative, and restricting the forms to V xo , we obtain 

(3.37) = a{x )Q A)X0 + (3(x )Q B>X0 for all x G M . 

Notice that /3(x ) ^ 0, since Qa,x 0> i n view of Lemma l3~2l 

Applying next the same kind of reasoning as in the proof of Lemma 13.91 only with 
V( Xt y) replaced by V x and Qa,x, v by Qa,x, etc., and m = 2, we see that ()3.37|) implies that 
there is a non-trivial linear combination D = aoA + (3qB such that 

(3.38) Qd,x = for all x G A^ - 
Notice also that 

(3.39) span K A/o = v . 

Indeed, otherwise A/o would be an open subset of a linear subspace W of dimension n — 2, 
and Qa|vk = 0. But this would imply rank A < 4, contradicting our assumption on A. 
Arguing similarly as in the proof of Lemma l3.10l ()3.39|) implies that 

(3.40) (J V x spans z/ x , 

for every non-empty open subset U of Af . Indeed, otherwise [J x€U V x would be contained 
in a linear subspace W of dimension n — 2, so that, by comparing dimensions, V x — W 
for every x G U. But this would imply Ao C W, contradicting ()3.39|) . 

Finally, we can apply a similar reasoning as in the proof of Lemma 13.111 in order to 
conclude that 

(3.41) QdU=0. 
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By Lemma f3. 21 this implies rank.0 < 2, hence rankD = 2, and thus minrank {A, B} = 2. 

To prove ([3.41)1 . we may assume that there is an orthonormal basis ei(x), . . . , e n _2(x) 
of V x , varying analytically in x G U. We then put 

fi(x) := Ax, f 2 (x) := Bx, 

so that 

ei(ar), . . . , e n _ 2 (x), f\(x), f 2 (x) 
is a basis of lR n . Consider the mapping F : U x IR™~ 2 — > v L , 

n-2 

F(x, z) := z jZj(x) e C!/ 1 , Z6f/,z6 IR n ~ 2 , 
i=i 

where we consider [/ as an analytic submanifold of W 1 of dimension n — 2. 

We shall prove that there is some (xq, zq) such that 
(3.42) rank DF(x , z ) = n - 1 . 

This implies that F is a submersion near (xo, zb), so that IJxgc/ ^ contains a non-empty 
open subset f2 of v^. Since, by ()3.38jl . Qb vanishes on Q, we thus obtain (|3.4ip . 
In order to prove ()3.42|) . we consider here, for x, z fixed, the linear mappings 

^ := V X ^R, 
MO ■= fi(x)-(D x F(x,z)(£)), 2 = 1,2, 

and 

»:=(£):V.-R». 

Similarly as in the proof of Lemma 13.111 it then suffices to show that linear mapping *f> 
has rank 1, generically. But, 

%A.F(x,z)) \ 
\B-F{x,z)) J^' 

and, for x G U, there is some z G R n_2 such that ipi\v x 7^ 0. Indeed, otherwise we would 
have (Av) • £ = for all v G V x , £ G V x , and in particular Qa,x = 0. But then rank 
A < 2 • 2 = 4, a contradiction. This shows that ip\ ^ 0, hence rank\l/ = 1. 

Q.E.D. 

Our main result Theorem 11.51 concerning the form problem is now an immediate con- 
sequence of Theorem 13.51 Theorem 13.121 and Lemma 13.131 

Remark 3.14 The statement of Theorem \1.51 remains true, if we replace the canonical 
symplectic form on M. 2d by an arbitrary constant symplectic form to, and define the Poisson 
bracket accordingly by 

{f,g} u :=u>{H?,HZ), 

where HJ denotes the Hamiltonian vector field associated to f (see Section^), and define 
Cby 

Qc '■= {Qa, Qb}ui- 

To see this, notice that there is a linear change of coordinates which transforms u into 
the canonical symplectic form on M? d , and that the space of quadratic forms and the cone 
of positive-semidefinite forms remain invariant under such a change of coordinates. 
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4 Applications to non-solvability of doubly charac- 
teristic differential operators 

We are now in a position to prove Theorem II .21 Assume that L satisfies the assumptions 
of this theorem at (#o>£o) ^ ^ e wor k m a new set of symplectic coordinates z : = 
(x — x ,(,— £ ) near (x ,^o)- In these coordinates, (x ,Co) corresponds to the origin. Since 
Dpk(0) = in these coordinates, a Taylor expansion of pk then gives 

(4.1) p k (z) = ^Hz + 0(\z\ 3 ) asz^O, 

where H = D 2 pk(xo,C,o) = A + iB. Writing / := Repk and g := Impk, this implies 

(4.2) {f,g}(z) = {Qa,Qb}(z) + 0(\z\ 3 ) = Q c (z) + 0(\z\ 3 ). 

Now, A and B satisfy the assumptions of Theorem 11.51 so that we can find some 
v G R 2n such that 

(4.3) Q A (v) = Q B {v) = and Q c {v) ^ . 
Notice that Qc( v ) = cr(2J 2n Av , 2J 2n Bv), where J 2n = ( % ) > so that in partic- 



N -In t 

ular Av and Bv are linearly independent. For i e 1 small and w G M 2n put h(t,w 
t~ 2 (f(t(v + w)),g(t(v + w))\ if t ^ 0. According to (fOjl . IjOJl . we have 



/i(t, w) = (2 \Av)w + Q A (w) + w), 2 '(Bvjtw + Q B (w) + i> 2 (t, *»)), 

where ipj(0,w) = 0. In particular, h extends smoothly for t = 0. Since h(0, 0) = and 

d w h(0, 0) = 2 f ) * s non-degenerate, by the implicit function theorem there is a 

smooth function t i— > to(i) on a small neighborhood / of the origin with to(0) = 0, such 
that h(t,w(t)) = 0. 

Putting ,z(t) := t(v+w(t)), this means that f(z(t)) = = g(z(t)) for every t G J\{0}. 
Moreover, by (|4.2j) . we have 

{/, <?}0<t)) = i 2 Q c (^ + u;(t)) + 0(t 3 ) = t 2 Q c (w) + 0(t 3 ). 

Since Qc{v) ^ 0, we thus see that for t ^ in I sufficiantly small, we have {/, g}(z(t)) ^ 
0. Thus, Hormander's condition is satisfied at z(t). If we re- write z(t) in the original 
coordinates as (x(t),£(t)), this means by Theorem 11.11 that L is not locally solvable at 
x(t). Since x(t) converges to xo as t — > 0, we thus see that L is not locally solvable at xq. 
This finishes the proof of Theorem 11.21 

Q.E.D. 

Next, we show how Corollary 11.41 follows from Theorem 11.21 Recall that p 2 {x,^) = 
t q(x,^)A(x)q(x,^). If we write z := (x ,^ ), since q(z) = 0, the Hessian form of L at z is 
thus given by 



Q H (w) = t wD 2 p 2 (z)w = 2 t w[ i TA(x )T}w, we 



r>2n 
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where 

T := Dq{z) : IR 2 "' -> R m . 

This means that 

(4.4) A = 2 Ti(x )T, 5 = 2 *rS(x )T, 

if we decompose W = A+iB as in Theorem 11.21 Since we are assuming that ({g,-, ?fe}(^)) j . fe _ 1 
is non-degenerate, the linear mapping T must be onto, so that 

(4.5) rank (a A + (3B) = rank (aA(x Q ) + (3B(x )) 
for every a, (3 G R, 

Next, from (|4.4|) . by some straight-forward computation one obtains that C, defined 
by Qc ■= {Qa, Qb}, is given by 

C = 4(2 Ti(x ) J (2) P(x )T + 2 *TB(x Q ) J w 1(x )t) , 

and similarly we have 

{Qi(x )'^o)}wW = 4U(zo)J W S(x )i;, f G lR m . 
Combining these results, we find that 

(4.6) C = A<TC(z)T, 

if C(z) is defined by Q C { Z ) ■= {Qa( X0 )iQb( X0 )}(z) as in Corollary O 

Observe next that (TJ 2n i T)jk = Vqj(z) JjntVg^z)) = {(^, gjfc}(^), so that 

(4.7) TJ 2n i T = J {z) . 

This implies tO( z \(Ji z )V, J( z )w) = cr(Tt>, Tiu) = a{J 2n *Tv, J2n t Tw), hence 

(4.8) uj( z }(v,w) = a(Rv, Rw), 
if we put 

R := J2n t TJ7 z ] ■ K m M 2n . 

Notice that 

TP = Idjjm. 

Moreover, if we set P := RT = J 2n i TJ7'}T : M 2n -> M 2n , then P 2 = P, so that P is a 
projector, and TP = T, hence 

R 2n = P{R 2n ) © (J-P)(M 2n ) 

(4.9) = P(M m )©KerT. 

Finally, one checks that J 2n P = t PJ2n, hence 

a{X,PY) = a(PX,Y), 
so that the decomposition in ()4.9|) is orthogonal with respect to a. 



43 



From (|4.8j) . (|4.9|) we see that a subspace V C M m is symplectic with respect to U( z ) 
if and only if the space T _1 (V) = R(V) © KerT is symplectic with respect to a. In 
combination with (|4.4|) . ()4.5|) and ()4.6|) this implies that the operator L satisfies the 
hypotheses of Theorem 11.21 which proves Corollary 11.41 

Q.E.D. 

As an application of Corollary 11.41 consider a connected, simply connected two-step 
nilpotent Lie group G. Up to an automorphism, such a group can always be realized as 
G = M. m x W (as a manifold), with a group law of the form 

0, u) ■ 0', v!) = (z + z',u + u' + i( *zJ(V)i) , 

for G lR m ,M,-u' G M.^, where J^\...,J^ are skew-symmetric m x m-matrices. A 
basis of the Lie algebra q of left-invariant vector fields is then given by 



J i=l 

U k := ^- fc=l,...,£. 



Consider an operator L of the form 



(4.10) L = QjkXjXk + lower order terms, 

j,k=i 

where the coefficient matrix A = {otjk)j.k £ Sym (m, C) is symmetric. We put A := He A, 
B : = ImA If we split £ = ((, /i) G lR m x K according to the coordinates a; = (z, u) G G, 
we easily compute that 

■J((*,u),(C,aO) = J ^ ' 
if we put J^ 1 := Finally, we define 

i=l 

:= 2(AJ fM0 B - BJ^A) . 
From Corollary II .41 we then immediately obtain 

Corollary 4.1 Assume that is non- degenerate for some fiQ G M*, that A,B forms a 
non-dissipative pair, and that A, B and C m are linearly independent. Moreover, suppose 
that either 

(i) minrank {A, B} > 3 and maxrank {A, B} > 17, or 

(ii) minrank {A, B} = 2, maxrank {A, B} > 9, and that the joint kernel ker Anker B of 
Qa and Qb is either trivial, or a symplectic subspace with respect to the symplectic 
form on M. 171 associated to *(J M0 ) _1 . 
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Then the operator L in (I4.1U)) on G is nowhere locally solvable. 



J 



In the special case of the Heisenberg group H^, we have m = 2d, I = 1 and : 
- ( ^ T ) ■ Then J M = /iJ, ji E R, so that we may choose /i = 1. Putting 



C := 2(AJfi - BJA) 



we obtain 



Corollary 4.2 // G = is the Heisenberg group, assume that A, B forms a non- 
dissipative pair, and that A, B and C are linearly independent. Moreover, suppose that 
either 

(i) minrank {A, B} > 3 and maxrank {A, B} > 17, or 

(ii) minrank {A, B} = 2, maxrank {A, B} > 9, and that the joint kernel ker Anker B of 
Qa and Qb is either trivial, or a symplectic subspace with respect to the canonical 
symplectic form on M. 2d (associated to — J). 

Then the operator L in ()4.10|) on M.^ is nowhere locally solvable. 

This shows that local solvability of L on Heisenberg groups can essentially only arize 
if the operator e i0 L is dissipative, for some 9 G R. This statement is true in the strict 
sense, if, e.g., the matrix A is non-degenerate, and d > 9. 

As we had already mentioned, the examples in [KM] and [MP] show that the analo- 
gous statement is wrong on Heisenberg groups of low dimension 5 and 7. 

Remark 4.3 Corollary \1.J\ applies also to higher step situations. 

For instance, assume that g is a nilpotent Lie algebra of step r > 3, and let S = 0i C 
02 C ••• C Q r +i = {0} denote the descending central series, i.e., := Let 
G = expg be the associated nilpotent Lie group, and choose elements Xi, . . . ,X m of g 
which form a basis modulo g2- Consider the Xj as left-invariant vector fields on G as usual, 
and let L on G be given by (J4.1(Jj) . Put Gj := expQj, and let H := G 3 \G = {G 3 g : g E G} 
denote the quotient group of G by G 3 . Then if is a 2-step nilpotent Lie group, and if we 
assume that also the lower order terms in (j4.10j) are left-invariant, then L factors through 
H ClS db left-invariant differential operator 



Here, Xj is the left-invariant vector field on H corresponding to Xj. 

Choose further elements X m+ i, . . . , Xn such that X±, . . . , X^ forms a basis modulo 
g3, and then elements Yx, . . . , so that X±, . . . , Xn, Yx, . . . , Yj» forms a basis of g. We 
may then choose coordinates (x, y) E R x IR fc of G, by putting 




lower order terms . 



j,k=i 



g(x,y) := exp \ yjYj exp \xtX t 
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Let (£,77) denote the dual variables. By 2na(A) we denote the principal symbol of a 
differential operator A. Then one easily shows that 



hence 



(4.11) 



a(L)((x,0),(£,0)) 



<t(L)(x,£) forallfeR 



N 



(here, we have chosen x as natural coordinates for H). Moreover, since 



[Xj, X k ] 




we have 



{a(Xj), a(X k )}{(x, 0), (f , 0)) = ia([Xj, X k ]){(x, 0), (f , 0)) 
= ia([Xj,X k ]~)(x,0 = {a(Xj),a(X k )}(x,0 . 



If J(x,£) denotes the skew symmetric matrix on H given in Corollary |HH and J(( x ,o),(5,o)) 
the one on G, we thus have 



Thus, if we assume that L satisfies the hypotheses of Corollary 14.11 then with L, also 
L satisfies the assumptions of Corollary 11.41 so that L is nowhere locally solvable. 



The proof of Proposition 12 .41 is a slight modification of a geometric argument suggested 
by Christoph Bohm, which has led to a much shorter proof of this result, compared to my 
original, analytic argument. I wish to express my thanks to him for providing his proof. 
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